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Overview 



In 2003, Lumina Foundation for Education launched a bold, multiyear, national initiative called 
Achieving the Dream: Community Colleges Count, to help students stay in school and succeed. 
The initiative is focused particularly on students who have faced the most barriers to success, 
including low-income students and students of color. Initially, 27 colleges in five states joined 
the initiative; there are now over 80 institutions in 15 states. 

Participating colleges commit to using data to improve programs and services in ways that lead 
to increased student success — a process known as “building a culture of evidence.” 
Specifically, colleges mine transcripts and gather other information to understand how students 
are faring over time and which groups need the most assistance. Based on a diagnosis of the 
problems in student achievement, they design and implement strategies to improve academic 
outcomes. Participating colleges receive a $50,000 planning grant followed by a four-year 
$400,000 implementation grant, along with assistance from coaches hired by the initiative. This 
report describes the progress made by the 13 Pennsylvania and Washington State community 
colleges that comprise Round 3 of the Achieving the Dream initiative after planning and one 
year of implementation. The key findings are: 

• The average institutional rates for Pennsylvania and Washington colleges on most 
of the baseline performance measures were low, and there was greater variation 
among colleges within the two states than between them. 

• There was widespread support among college leaders and other personnel for the 
Achieving the Dream goals and principles, which were seen as consistent with 
college goals and accreditation and state accountability requirements. 

• All 13 colleges used an analysis of their college’s data as the primary means of 
identifying gaps in student achievement, and all used both qualitative and 
quantitative data to identify and prioritize problems areas. 

• The strategies developed by the colleges focused on four areas: developmental 
education, supplemental instmction, a first-year student success course, and better 
organized and more intensive advising. 

• Four colleges were beginning to institutionalize a culture of evidence, and another 
four had made promising progress after the first year of implementation. Five had 
made little or only limited progress. 

• Achieving the Dream had positive effects on all of the 13 Pennsylvania and 
Washington State colleges, which as a group were further along a year and a half into 




the process than were the colleges that joined the initiative two years earlier in the first 
round. 

The findings from this study will be compared with follow-up research that CCRC and MDRC 
will conduct in two years to evaluate the progress of the colleges at the end of the five-year 
project period. 
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Preface 



With their open admission policies, convenient locations, and low tuition, community 
colleges are a critical resource for millions of adults who might otherwise be unable to go to 
college. For low-income people in particular, these colleges offer a pathway out of poverty and 
into better jobs. Yet nearly half of all students who begin at community colleges do not transfer 
to a four-year college or complete a certificate or degree program within eight years of initial 
enrollment. 

Can community colleges make better use of data to improve student outcomes? That is 
the fundamental idea behind Achieving the Dream: Community Colleges Count, a bold initiative 
launched in 2003 by Lumina Foundation for Education to help community college students 
succeed — particularly low-income students and students of color, who have traditionally faced 
the most barriers to success. Today, Achieving the Dream includes over 80 colleges in 15 states, 
supported by 7 partner organizations and 21 funders in addition to Lumina. The initiative’s 
central focus is to help community colleges use what they learn from data on student outcomes 
to develop new programs and policies — and to generate long-term institutional change. 
Achieving the Dream provides a way for colleges to engage in thoughtful self-assessment and 
reflection on how they can serve students better. 

This report, a coproduction of the Community College Research Center (CCRC) at 
Columbia University’s Teachers College and MDRC, presents the findings from baseline 
evaluation research on the 13 colleges that comprise the third round of the Achieving the Dream 
initiative. Findings focus on the initial efforts of seven Pennsylvania and six Washington State 
community colleges to build a culture of evidence for student success and, more specifically, on 
their work to increase the equity of achievement among students by race and ethnicity or by 
income. The report also compares the Pennsylvania and Washington State colleges with the first 
27 community colleges that joined the initiative. 

This report reflects the implementation to date by the Pennsylvania and Washington 
colleges early on in a five-year process. We will continue to investigate whether and how 
colleges make changes in their organizational culture and practices to serve students more 
effectively, examining especially whether outcomes improve on such critical measures as the 
rates of students who complete developmental education courses and who persist from semester 
to semester. 



Thomas Bailey 

Director, Community College Research Center 

Thomas Brock 

Director, Young Adults and Postsecondary Education Policy Area, MDRC 
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Executive Summary 



Introduction 

Traditionally, community colleges have played a vital role in American society by 
expanding access to a college education for millions of Americans. In recent years, community 
college educators, under pressure from government agencies, accreditation agencies, and 
students themselves, have begun to pay more attention to what happens to students once they 
enter college and to take steps to increase the rates at which community college students earn 
college credentials and transfer to baccalaureate institutions. 

The Achieving the Dream initiative 

One of the most important initiatives in this shift in community college attention from 
access to access and success is Achieving the Dream, a national initiative involving more than 
80 colleges in 15 states. The initiative seeks to help more community college students succeed 
and is particularly concerned about students of color and low-income students, who traditionally 
have faced significant barriers to success. Whereas most efforts to improve community college 
student success involve specific programmatic interventions. Achieving the Dream is based on 
the premise that to improve outcomes for students on a substantial scale, colleges need to 
change how they do business in fundamental ways. Specifically, colleges should create a 
“culture of inquiry and evidence” in which decisions about the design, delivery, and funding of 
programs and services are made based on evidence of what works to improve student outcomes. 
Colleges that operate in this way adhere to four principles: (1) Committed leadership; (2) Use of 
evidence, specifically data on student progression and outcomes, to improve programs and 
services; (3) Broad engagement of administrators, faculty, staff, and students in efforts to 
promote student success; and (4) Systemic institutional improvement. 

Achieving the Dream recommends that colleges transform themselves according to 
these principles and thereby build a culture of evidence through a five-step process: (1) Commit 
to improving student outcomes; (2) Use longitudinal student cohort data and other evidence to 
identify and prioritize problems in student achievement; (3) Engage faculty, staff, and other 
internal and external stakeholders in developing strategies for addressing priority problems; (4) 
Implement, evaluate, and improve strategies; and (5) Institutionalize continuous improvement 
of programs and services through program review, planning, and budgeting processes driven by 
evidence of what works best for students. 

Achieving the Dream expects that by following this institutional transformation 
process, colleges will be able continuously improve rates of student success, including increased 
course pass rates, persistence, and, ultimately, credential attainment. 
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Achieving the Dream provides both financial and technical support to help colleges 
undertake this process. The financial support includes a one-year planning grant and 
implementation funding over four years that colleges can use to support data collection and 
analysis, engagement of faculty and staff, and implementation of improvement strategies. The 
technical support includes two outside consultants — a coach (usually a former community 
college president) and a data facilitator (usually a community college institutional researcher) 
— who advise the college on how to analyze its data on student success, interpret and 
communicate the findings to faculty and staff, and use the information to make improvements in 
college programs and services. 

The Round 3 Colleges 

Thirteen colleges, seven in Pennsylvania (PA) and six in Washington (WA) State, 
joined Achieving the Dream in 2006 in the third round of entering colleges (Table ES.l). All of 
them participated in a planning year that included a Kickoff Institute in July 2006 and produced 
proposals that were accepted for four years of implementation funding. 

Achieving the Dream: Community Colleges Count 
Table ES.l 

Achieving the Dream Colleges in Pennsylvania and Washington State 
Selected Characteristics, Academic Year 2005-06 



College 


Enrollment (FTE) 


Minority 
Enrollment (%) 


Pell Recipients 

(%) 


Pennsylvania 


Allegheny County 


12,443 


28 


34 


Beaver County 


1,886 


20 


37 


Delaware County 


3,664 


29 


21 


Montgomery 


5,684 


31 


18 


Northampton 


4,525 


22 


23 


Philadelphia 


13,542 


68 


54 


Westmoreland 


4,116 


11 


40 


Washington State 


Big Bend 


1,464 


29 


44 


Highline 


4,635 


48 


18 


Renton Technical 


2,782 


51 


27 


Seattle Central 


4,912 


47 


21 


Tacoma 


5,064 


39 


30 


Yakima 


3,592 


38 


40 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Integrated 
Postsecondary Education Data System (IPEDS). 
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The Evaluation 



The Community College Research Center (CCRC) and MDRC conducted baseline 
evaluation research to examine efforts by the 13 Achieving the Dream colleges in Pennsylvania 
and Washington to begin implementing the initiative’s institutional improvement process during 
the planning and first implementation year. Specifically, the researchers sought to determine the 
following: what was the performance of the colleges at baseline; how closely the colleges 
followed the improvement process recommended by Achieving the Dream; what student 
success strategies the colleges were implementing and what were the results to date; how much 
progress the colleges made in building a culture of evidence; what effects Achieving the Dream 
had on the colleges early on in the initiative; and, finally, how the colleges and the initiative 
more generally can improve the impact of their efforts moving forward. 

Findings based on extensive on-site interviews with personnel at all 13 colleges, a 
survey of data use by faculty and administrators at these colleges, and an analysis of data on the 
performance of the colleges in the period before they joined the initiative are presented below. 
Findings for the PA and WA colleges are compared with each other and with findings from a 
baseline evaluation of the 27 colleges that joined the initiative in the first round, which was also 
conducted by CCRC and MDRC. The findings from this study will be compared with follow-on 
research that CCRC and MDRC plan to conduct in two years to see what progress the PA and 
WA colleges have made by the end of their five-year project period. 



The Baseline Performance of the 
Pennsylvania and Washington Colleges 

At the beginning of the initiative. Achieving the Dream established five main 
performance indicators, with specific student achievement measures for each, for participating 
colleges. To establish the baseline performance of the PA and WA colleges on the Achieving 
the Dream measures, we calculated the average performance of the PA and WA colleges on 
each measure for the three-year period before each college joined Achieving the Dream using 
data on cohorts of first-time, degree-seeking students that the colleges participating in 
Achieving the Dream are required to report to a national database maintained by the initiative. 

The average institutional rates for PA and WA colleges on most of the baseline 
performance measures were low, as they were for the Round 1 colleges. Interestingly, while 
there was variation in the average performance rates for WA, PA, and Round 1 colleges on all 
of the Achieving the Dream measures, there was often more substantial variation within these 
three groups than among them. 
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Course Completion 

• Developmental courses. PA colleges had a higher average rate of successful 
completion for developmental instmction in all three subjects (math, English, 
and reading) than WA colleges. PA college rates did not vary as widely as in 
WA, however. Both PA and WA colleges had higher average rates of 
completion for developmental English than did Round 1 colleges, but Round 
1 colleges had a higher completion rate than both PA and WA in 
developmental reading. 

• Gatekeeper courses. Rates of completion of the first college-level 
“gatekeeper” courses in math and English are important because passing 
these courses is associated with a higher likelihood of earning college 
degrees and transferring. PA and WA colleges had higher average rates of 
completion in gatekeeper English courses than they did in college-level math 
courses, and the average rates at which students completed gatekeeper 
English were higher for students who were referred to developmental 
instruction than for students who were not. Both PA and WA colleges had 
higher average rates of completion in both math and English gatekeeper 
courses than did Round 1 colleges. 

• Overall course completion. The average course completion rates for PA, 
WA, and Round 1 colleges were very similar, slightly more 75 percent, but 
PA colleges had a much larger range in variation than WA colleges. 

Persistence and Credential Completion 

• Persistence over three years. As would be expected, the average rates of 
persistence decreased as the period of time from initial enrollment increased. 
WA colleges had the highest percentage of students persisting across the 
three measured periods of time; moreover, as time passed, the gap between 
WA colleges’ rates of persistence and both PA and Round 1 colleges’ rates 
of persistence increased. 

• Credential completion. PA colleges’ average rate of credential completion 
closely matched the Round 1 colleges, while WA’s average rate was higher. 
WA also had higher rates of obtaining an associate degree within three years 
than did either the PA or Round 1 colleges. 
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Pell Status 



WA exhibited higher average rates of completion within three years for both Pell 
recipients — low-income students who receive federal needs-based grants — and non- 
recipients than did PA and Round 1 colleges. Consistent with Round 1 colleges, rates of 
persistence for PA and WA colleges were higher for Pell recipients than non-recipients. This 
may stem in part from the fact that Pell Grant recipients are encouraged to attend college full- 
time and full-time students are not surprisingly more likely to graduate than part-time ones. Pell 
recipient rates of credential completion were low for all three groups, however. 

Race and Ethnicity 

The average institutional rates for successful completion of developmental and 
gatekeeper courses were lower for African-Americans, Hispanics, and Native Americans than 
for whites, with PA colleges having more gaps on these measures than WA colleges. In PA, all 
of the minority groups had lower average rates than whites for completion of gatekeeper math 
and English courses. In WA, Asians, African-Americans, Hispanics, and Native Americans all 
had higher rates of completion in gatekeeper English, though not in math, than whites. Across 
both PA and WA colleges, the rates at which students completed courses generally were lower 
for African-Americans, Hispanics, and Native Americans than for whites — a gap also present 
among the Round 1 colleges. 



Patterns of Data Use by Faculty 

In late 2008, CCRC and MDRC conducted a survey to identify patterns of data use by 
faculty and administrators at the Achieving the Dream colleges. The main findings are 
summarized below. 

Extent of Data Use 

Overall, a surprisingly high proportion of faculty in the PA, WA, and Round 1 colleges 
regularly used data on student outcomes, although there were variations across and within 
colleges on the types of data used most often. 

• Frequency. At least once a year, about half of the faculty across all the 
Achieving the Dream colleges used data on placement test scores, retention 
rates, or graduation rates, and used measures of student learning other than 
grades, although over a third never used such measures. 

• Teaching-related decisions. The majority of faculty surveyed used data and 
research at least to some extent in decisions related to teaching. Around one 
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in five indicated that they were a heavy user of data and research for teaching 
decisions. 

• Consideration of student achievement gaps. Nearly one in three faculty 
respondents never reviewed data on student achievement gaps among 
different student groups, although WA college faculty reviewed such data 
more frequently than their PA counterparts, possibly because their students 
included a higher proportion of minorities. Faculty at the PA and WA 
colleges were significantly more likely than those at Round 1 colleges to 
indicate that they participated frequently in organized discussions about 
improving the academic performance of students of color. 

• Academic department decisions. Most faculty indicated that their 
departments used data and research for programmatic decisions at least to 
some extent, and the departments of approximately one fourth were heavy 
users of data. However, the frequency with which faculty in the PA, WA, 
and Round 1 colleges used data for decision making varied by department, 
with those in general education on average less likely to use data on student 
outcomes in their work, while faculty in developmental and for-credit 
occupational programs were more frequent users of data and research. 

• Effect of departmental vs. college-wide practices. Interestingly, we found a 
much stronger relationship between data use by individual faculty and the 
extent to which their department used data on students for decision making 
than between faculty data use and the extent to which the college overall 
used data on student outcomes to evaluate programs and make decisions at 
the leadership level. Hence, commitment by top college leaders to data-based 
decision making and a data-oriented approach to institutional management 
may not be sufficient to encourage faculty to become more data oriented in 
practice. Additional efforts at the department level are probably needed to 
change faculty behavior. 

Accessibility of Data and Training in Its Use 

A majority of faculty at the PA, WA, and Round 1 colleges indicated that they were 
able to access information they needed in a timely manner and that the information they 
received was accurate, although faculty from the WA colleges were less satisfied with their 
access to data, possibly because of the problems that the WA community and technical colleges 
had retrieving data from the legacy information system they shared. 
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• Methods of data retrieval. Faculty indicated that they used a variety of 
sources or methods to get information on groups of students. WA college 
faculty were significantly less likely than PA and Round 1 faculty to do 
searches themselves using their college’s student information system or their 
college’s website or fact book because of retrieval problems. 

• Support from the institutional research staff. Faculty at about half of the PA 
and Round 1 colleges indicated that their college’s institutional research (IR) 
function was adequately staffed to meet the demand for information, 
compared with a third of WA college faculty. PA college faculty were 
significantly more likely than those in WA and Round 1 colleges to indicate 
that their college’s institutional research staff was responsive to requests for 
information. At least some colleges had trouble recruiting qualified IR staff. 

• Perceived barriers to use. Around a third of the faculty at the PA, WA, and 
Round 1 colleges indicated that one reason that they did not use data and 
research was that they were too busy with their teaching responsibilities. 

Most faculty, however, indicated that using data and research on students 
was part of their responsibility and that they had the skills needed to analyze 
data. About a fourth of faculty said that the data available were not relevant 
to their jobs. 

• Training for data use. The percentage of faculty who indicated that they had 
been involved in training or professional development on institutional 
research or data analysis in the past year ranged from 28 percent for the WA 
college faculty to 39 percent for the Round 1 college faculty. Over half of the 
faculty at the PA, WA, and Round 1 colleges said that they participated in 
training or professional development on program evaluation or assessment. 

While faculty who had recently participated in training or professional 
development in either of these topics were more likely to use data in their 
work, this finding does not necessarily mean that colleges could increase data 
use by increasing the amount of training provided, since it is possible that 
faculty and administrators who were heavier users of data were more likely 
to seek out training in data use. 

Possible Effect of Achieving the Dream on Data Use 

Not surprisingly, faculty and administrators who participated in Achieving the Dream 
activities were significantly more likely to use data on student outcomes than were those not 
involved in the initiative. Moreover, faculty at the Round 1 colleges were significantly more 
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likely than those in the PA and WA colleges to indicate that they use data on retention and 
graduation rates frequently. This is consistent with the hypothesis that colleges that have been 
involved in Achieving the Dream longer should be more advanced in their use of data for 
improving student success. However, neither finding can be seen as definitive evidence of a 
causal relationship between Achieving the Dream and more extensive use of data for 
improvement. CCRC and MDRC will have better evidence with which to examine the effect of 
Achieving the Dream on data use when we conduct a follow-up survey of faculty and 
administrators in the WA and PA colleges in two years, near the end of their participation in the 
initiative. 



College Progress on Institutional Improvement 
in the Planning Year 

During the planning year. Achieving the Dream colleges are expected to begin carrying 
out the first three steps of the initiative’s five-step institutional improvement process, which are 
designed to engage college personnel in identifying areas where students are experiencing 
barriers to success and designing strategies to break down those barriers. 

Commit to Improving Student Outcomes (Step 1) 

This first step calls for the college’s leadership to make a clear commitment to improve 
student outcomes, not just to increase enrollments. 

• Senior leadership commitment. Across all 13 PA and WA colleges, college 
leaders demonstrated a willingness to reallocate resources to improve student 
outcomes, including the hiring of additional institutional researchers. Eleven 
of the 13 college presidents were actively engaged in Achieving the Dream 
activities and were visible advocates for the initiative on their campuses, 
including regular participation in core team planning. (The core team was to 
include the college’s president, vice presidents or deans for academic affairs 
and student services, a faculty representative, and a person responsible for 
institutional research or effectiveness.) Most presidents — a larger 
percentage than Round 1 college presidents — tapped members of their 
cabinets or executive teams to lead the implementation of the initiative, and 
they all kept their board of directors regularly updated on initiative activities 
throughout both the planning year and the first implementation year. 

• Incentives for leadership commitment. None of the colleges considered 
grant money as an incentive for participation in Achieving the Dream. 
Rather, they identified the following as incentives: (1) consistency with 
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previously-identified college goals; (2) involvement with a high-profile 
national student success initiative, which lent prestige to the college and 
allowed conversations with faculty and staff about student outcomes without 
creating the perception that the administration was blaming the faculty for 
poor student outcomes; (3) provision of a roadmap to achieve the goals of 
improving outcomes and closing the achievement gap; (4), synergy with 
accreditation standards, which would help their college prepare for 
compliance through the development of the culture-of-evidence approach to 
institutional improvement; and (5) alignment with state higher education 
goals and performance accountability requirements. 

• Internal college communication about Achieving the Dream. The PA and 
WA presidents and senior administrators used a variety of methods to inform 
the college community about the initiative, including college-wide forums 
such as fall convocations, faculty in-services and other professional 
development days, email alerts, data briefs, and featured presentations by 
Achieving the Dream coaches and data facilitators. In over half of the 
colleges in both PA and WA, faculty and staff interviewed by the research 
team suggested that a substantial number of their colleagues understood both 
the goals and the details of the initiative. 

• Organization and management of the initiative. All of the colleges began 
their Achieving the Dream work with a core team, which generally involved 
representatives of a broad cross-section of college personnel, including 
faculty leaders, mid-level administrators, and student services staff. All but 
two colleges also began the planning year with separate data teams, and, with 
one exception, they included non-IR personnel. One of them started its 
planning year with a combined core and data team and the other created not 
just one data team, but a team for each of the five main Achieving the Dream 
performance indicators. Other strategies used by the colleges to promote 
support for the initiative were the engagement of faculty and faculty union 
leaders in core team activities and the rotation of the core team membership 
to facilitate understanding of the initiative and participation among a broad 
segment of the college. 

Use Data to Identify and Prioritize Problems (Step 2) 

Step 2 of the Achieving the Dream process of building a culture of evidence calls for 
the colleges to use longitudinal student cohort data and other evidence to identify gaps in 
achievement among different student groups as well as “leakage points” where students struggle 
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or drop out. A key assumption of this approach is that once faculty and staff see that certain 
groups of students are not doing as well as others, they will be motivated to address barriers to 
student success. 

• Process for identifying achievement gaps. All 13 colleges relied on an 
analysis of their own college’s data as the primary means of identifying gaps 
in student achievement, though the majority had not done so before joining 
the initiative. Twelve used longitudinal cohort analysis to identify problems, 
and all the colleges disaggregated their data analyses by student race and 
ethnicity to identify achievement gaps. The colleges collected qualitative data 
to identify problem areas through both student and faculty focus groups and 
student surveys. In contrast, only about half of the Round 1 colleges used 
longitudinal cohort tracking as part of their analysis of student performance. 

• Institutional research capacity. Just over half of the colleges hired new staff 
for their institutional research offices. Two of the three colleges that did not 
have an IR department prior to joining the initiative established institutional 
research (or institutional effectiveness) offices. IR personnel turnover 
delayed the data collection and work of the data teams to various extents 
across the colleges, and several colleges had difficulty hiring IR staff. 

• Presentation of data analysis to faculty and staff. All 13 colleges presented 
the results of their analysis of achievement gaps to faculty and staff across 
their institutions using a variety of communication methods. While evidence 
of poor student performance caused some faculty to deny it was their 
responsibility (though fewer PA and WA college faculty did so than Round 1 
faculty), or to blame the students, in general such data was met with genuine 
interest and reflection by faculty and staff. Indeed, at every PA and WA 
college, faculty and staff indicated that the identified achievement gaps and 
problems areas in student outcomes provided motivation to improve and 
prioritize student success strategies. Round 1 college faculty were less 
motivated by such findings, and some were concerned that data on student 
performance would be used to penalize them. 

Engage Stakeholders in Developing Strategies for Addressing Priority 
Problems (Step 3) 

In Step 3 of building a culture of evidence. Achieving the Dream encourages the 
colleges to involve as many voices as possible in the process even though doing so could prove 
challenging for colleges already stretched thin serving disadvantaged students. The buy-in of 



ES-10 




faculty and staff on the front lines of working with students is critical for effective and 
sustainable student success interventions. 

• Receptiveness to the initiative. Faculty at the PA, WA, and Round 1 colleges 
generally had a favorable view of the initiative, particularly when adherence 
to its goals and principles supported efforts they were already making. 
Colleges where there is healthy collaboration between administrators and 
faculty and student services staff were more receptive to the initiative. 

• Concerns about Achieving the Dream. At almost half of the colleges, some 
faculty members were concerned about the time requirements of the 
initiative, particularly if it would be short lived. At several colleges, some 
faculty expressed concern that improving student success would mean 
lowering standards. 

• Process for designing strategies to address achievement gaps. Colleges 
largely followed the Achieving the Dream planning process in the design of 
new strategies and most did not develop improvement strategies until after 
analyzing their data. Teams from all 13 colleges participated in the 
Achieving the Dream Strategy Institute, which was also well attended by 
teams from previous rounds. Several colleges took note of mistakes and 
successes of these earlier round colleges, and many of the strategies adopted 
in WA and PA were informed by presentations at the Strategy Institute. In 
addition, college personnel at several institutions reported using the 
Achieving the Dream website as an additional resource to support strategy 
development. 

• Staff involvement in the planning process. Seven of the 13 colleges engaged 
faculty and staff on a fairly wide scale in the process of using data to develop 
student success strategies, a proportion comparable to that for the Round 1 
colleges. Yet, at the other 6 colleges a relatively small number of faculty and 
staff were actively involved in analyzing the data on student success and 
identifying strategies for improvement. Only 2 colleges gave faculty release 
time from instmction to facilitate their participation in initiative planning. For 
adjunct faculty in particular, scheduling and college expectations regarding 
their participation on campus committees or at meetings were barriers to their 
involvement with the initiative. 

• Board, student, and community engagement. College presidents kept their 
boards of tmstees regularly informed of initiative activities and a few 
colleges included board members on their core teams, but most board 
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members were not routinely engaged in the initiative. Similarly, while 
student focus groups contributed insights into problem areas at most colleges, 
no college chose to engage students directly in designing strategies. 
Community members or groups were rarely informed about the initiative or 
engaged in its activities. 



First-Year Implementation of Strategies 
for Improving Success (Step 4) 

In the fourth step toward building a culture of evidence, colleges begin implementing 
the strategies that they described in their implementation plans to evaluate the outcomes of their 
strategies and to use the results to make further improvements and scale up those that are 
successful. 

Prevalent Strategies 

The 13 PA and WA colleges, which had nearly completed their first year of a four-year 
institutional improvement process when the research team reviewed their progress, had 
developed strategies in seven broad categories that were similar to those developed by the 
Round 1 colleges: advising, developmental education, financial support, first year experience, 
high school and community outreach, professional development, and supplemental 
instruction/tutoring/study groups. The following four strategy types were most prevalent. 

• Developmental education. Twelve of the 13 colleges, like many of the 
Round 1 colleges, implemented at least one strategy that targeted students in 
developmental education courses. They involved the modification of 
academic policies, including the way that students were placed into 
developmental education; cohort-based learning and learning communities; 
curriculum restmcturing; and course revision and expansion. Defining 
learning outcomes for developmental courses and putting in place 
mechanisms for assessing outcomes was a more common strategy among the 
PA and WA colleges than those in Round 1. Since student success in 
developmental math was a particular concern, 11 of the 13 colleges pursued 
strategies that targeted students who placed into developmental math. 

• Supplemental instruction, tutoring, and study groups. Eight of the 13 PA 
and WA colleges, like a majority of the Round 1 colleges, developed 
strategies for providing students — most often developmental education 
students or students in gatekeeper courses — with additional learning support 
resources. Four of them implemented supplemental instruction in which peer 
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leaders attended classes and held review sessions for students. One college 
was expanding its online tutoring capacity to reach students who lived 
considerable distances from the campus; another was experimenting with 
“embedded tutoring,” in which a peer tutor shadowed stmggling students in 
their courses each day then helped them during after-class hours. 

• First-year experience. One strategy designed to provide students with a 
positive initial college experience, which research shows is critical to 
persistence and success, is to develop student success courses. These courses, 
prevalent among the PA, WA, and Round 1 colleges, are designed to help 
first-year students build the knowledge and skills needed to succeed at 
college, such as study skills, and time and financial management, to develop 
plans for college and careers, and connect with support services. 

• Advising strategies. Eight of the PA and WA colleges implemented at least 
one new advising strategy. Several colleges were targeting underrepresented 
students for enhanced student advising, including first-time college students, 
Hispanic students, ESL students, academically underprepared students, and 
low-income students. Several colleges also began considering mandatory, 
though short-term, advisement for some students. 

Colleges’ Progress in Strategy Implementation 

By the end of the first implementation year, all the PA and WA colleges had begun 
preliminary implementation of at least one strategy as part of Achieving the Dream, as the 
Round 1 colleges had at the same point in the process. 

• Strategies under development. Pour of the 13 colleges were still in the early 
implementation phase; the colleges had staff working on the strategies and 
were in the process of making preliminary steps toward implementation, but 
the majority of their strategies were still under development. Colleges at this 
level often expressed a need for additional research and planning time. Other 
colleges were reviewing potential changes in institutional policies. Several 
college strategies required additional training for staff involved. 

• Partial implementation. At 9 of the 13 colleges the majority of initiative 
strategies were partially implemented: they were still piloting strategies or 
were in the process of revising or modifying them. 

• Full implementation. Three PA and two WA colleges had at least one 
strategy that was fully implemented in that it had reached the college’s 
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proposed scale and target population. No college had a majority of its 
strategies fully implemented. Further, the few strategies that had been fully 
implemented were generally those with which the college had some 
experience in the past, those that represented a change in college policy or 
procedures, or were professional development activities for faculty and staff. 

• Scope of target population for strategies. Eight of the colleges had at least 
one or two strategies that were currently reaching large numbers of students: 
most concerned placement testing; alignment of developmental education, 
gatekeeper math, and English curricula; and ending late registration. Strategy 
implementation at the other colleges tended to still be in the early pilot 
stages, affecting a relatively small group of students thus far. 

Factors Affecting Strategy Implementation 

Several of the factors that influenced college progress in identifying student 
achievement gaps and developing strategies for addressing priority problems were also key to 
college progress in the implementation of initiative strategies. 

• Faculty engagement. Slightly more than half of the PA and WA colleges 
had successfully engaged faculty and staff in implementing initiative 
strategies, but most had difficulty initially in recruiting faculty, and, at one 
college, few faculty and staff were showing up for professional development 
activities, one of the college’s strategies. Some college faculty were hesitant 
to commit time and energy to what might be a temporary undertaking. 

• Student service staff engagement. At 6 of the 13 colleges. Achieving the 
Dream substantially increased student services involvement in student 
success efforts and at another group of 6 colleges the initiative strengthened 
collaboration between faculty and student services. At a few colleges, 
inadequate collaboration between faculty and student services staff hampered 
implementation. 

• Personnel turnover. Considerable turnover in key personnel, a factor that 
delayed the collection and data analysis for some colleges, also delayed 
strategy implementation at three of them. 

• Recruitment of students into strategies. At least three colleges reported 
difficulty recruiting students for their strategies, and a PA college delayed 
implementation of three learning communities because of insufficient student 
enrollment. 
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Evaluation of Strategies 

• Status of college evaluations. Four of the colleges had formal plans for 
evaluating their strategies, but only two had developed what the research 
team considered to be sound evaluation designs. Because many of the 
colleges had faced delays in implementing strategies, they had few 
evaluation results by the time of the research team visits in spring 2008. 

• Factors affecting the evaluation process. Several colleges had little prior 
experience in evaluating program outcomes, and they lacked the institutional 
research capacity to conduct high-quality evaluations of the strategies. At just 
over half of the colleges, overburdened IR staff and turnover among IR 
personnel hindered evaluation. Weak collaboration between IR and 
faculty/staff was also an issue, with several colleges piloting interventions 
without much thought about proper research design. 

Plans for Scaling Up Strategies 

With a handful of exceptions, few of the PA and WA colleges, like their Round 1 
counterparts at a similar stage in the initiative, had given much thought to bringing successful 
strategies to scale. Only two colleges appeared to have a plan for reaching more students. Most 
were still experimenting with small-scale strategies to see what worked. 

• Impediments to scaling up. Most colleges were not ready to scale up 
strategies because they did not yet know what worked. Several, which were 
under financial pressures or lacked discretionary funds, raised the question 
about the sustainability of their Achieving the Dream-supported strategies 
once the grant funding ran out. 



Progress Toward Institutionalizing 
a Culture of Evidence (Step 5) 

As of the time of our visits in spring 2008, the research team found that 4 of the 13 PA 
and WA colleges were beginning to institutionalize a culture of evidence on their campuses. 
Another 4 had made promising progress. The team found that 3 had made limited progress 
toward institutionalizing a culture of evidence, although major obstacles remained, and rated 2 
as making little or no progress. In comparison, fewer than half of the Round 1 colleges were 
making progress toward institutionalizing a culture of evidence at a similar stage of the project. 
The research team identified several factors that distinguished the leaders from the laggards: 
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• Leadership commitment. The president and other top administrators at 
leader colleges not only said that they were committed to student outcomes, 
they acted on their convictions, showing a willingness to make substantive 
changes in institutional policy and practice and to invest in resources 
necessary to support such changes. 

• Faculty and staff engagement. Leader colleges were more effective in 
involving faculty and student services staff in efforts to improve student 
success. 

• Staff collaboration. Collaboration between faculty and student services staff 
on student success efforts was stronger at leader colleges. Laggard colleges, 
conversely, often struggled to overcome the “silos” between academic and 
student affairs that often characterize community colleges generally. 

• Cross-division communication. Leader colleges were more likely to have in 
place committees for bringing together personnel from across the institution 
to work on student success. 

• A strong institutional research department. Leader colleges generally not 
only had the capacity to get the information they needed but IR staff was part 
of the management team. Some of the laggard colleges had strong IR 
departments, but they were not used strategically for improvement as they 
were in the leader colleges. 

• Evidence-based program review and planning. Leader colleges were more 
likely to have implemented evidence-based program review and strategic 
planning systems than were laggards, although having a strategic planning 
process was not sufficient to bring about changes in programs and services. 



The Impact of Achieving the Dream 

Some of the PA and WA colleges made more progress than others in moving toward a 
culture of evidence, and, indeed, the research team identified substantial progress at 8. 
Nevertheless, Achieving the Dream had positive effects on nearly all 13 of the PA and WA 
colleges involved. For some. Achieving the Dream provided a framework for analyzing data on 
student progression and outcomes that helped to focus college personnel on student 
achievement gaps and motivated them to find ways to address them. At several of the colleges, 
participating in Achieving the Dream helped to increase discussions about student success 
across the campus. 



ES-16 




Effects at the Colleges 

• Progress toward implementing a culture of evidence. The initiative helped 
the two PA and two WA colleges that made the most progress toward 
implementing a culture of evidence speed the transformation that they had 
begun even before joining the initiative. The three PA colleges and one WA 
college that made promising progress expanded their IR capacity: Three had 
no IR staff when they joined the initiative, but two created IR offices and the 
third organized faculty and staff into teams to examine the effect of college 
policies on student success and to recommend changes; and the existing IR 
office at the fourth college assumed a much more prominent role in efforts to 
improve student success. 

• Additional effects for all colleges. Even the five colleges with limited 
progress realized benefits from the participation in Achieving the Dream. 
Among all 13 colleges: (a) most saw the initiative as an “umbrella” for other 
student success initiatives; (b) more than half either added IR staff, purchased 
data analysis software, or upgraded their information systems; (c) half 
changed their committee stmcture to allow for a greater focus on student 
success; (d) 10 reported that the initiative helped them prepare for or comply 
with accreditation requirements; and (e) 10 colleges reported that the 
initiative helped them meet statewide performance accountability 
requirements. 

• Emphasis on equity. About half the colleges in both states developed student 
success strategies designed expressly to address gaps in achievement by 
race/ethnicity or income, with most basing them on analyses of student 
outcomes data that indicated gaps in achievement among minority or low- 
income students. Most of the colleges, however, did not attempt to make 
inequities in achievement a college-wide focus and priority, and personnel at 
some colleges expressed concern that targeting particular groups of students 
for special support was unfair to other students. 

The Value of the Achieving the Dream Supports 

• Coaches and data facilitators. These advisors were seen by most colleges as 
a particular strength of the Achieving the Dream initiative design. Many 
colleges saw their coach and data facilitator as a team and considered them to 
be mentors in the institutional change process. 
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• The Achieving the Dream database. Less than half of the colleges relied on 
this database in the initial analyses they conducted as part of the planning 
phase, instead using their own data. A few colleges planned to use the 
national database to compare their performance to other colleges, but the one 
or two colleges that tried to use the database in this way had difficulty doing 
so. 



• Strategy Institutes. In general, interview respondents who attended any of 
the annual Achieving the Dream Strategy Institutes found them useful. 
Several said that the opportunity to meet with colleagues from earlier-round 
colleges was particularly useful, and some indicated that they valued having 
time with colleagues from their own institutions. 

Suggestions for Improvement 

Increasing opportunities to learn what other colleges are doing was a common 
suggestion from the colleges, but interviewees also had other recommendations for the 
initiative: 

• Increase opportunities to share information with other colleges, so that 
they can learn about each other’s strategies and progress. 

• Increase the use of personnel from Achieving the Dream colleges as 
coaches for new colleges, to ensure that they have relevant knowledge and to 
enable colleges to benefit from earlier participants in the initiative. 

• Improve the availability of comparative performance data, so that the 
colleges can know how they are faring in terms of student outcomes. 

• Expand opportunities and support for faculty involvement, since engaging 
faculty is a challenge for most colleges. 

• Rethink Achieving the Dream plans for national expansion, which include 
a fee-for-service model that might not attract participation from colleges that 
do not believe that they have an achievement gap. 
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Chapter 1 



Introduction: Principles and Process 
for Improving Student Success and College Performance 

This report presents the findings from baseline evaluation research conducted by the 
Community College Research Center (CCRC) and MDRC on the initial work of community 
colleges in Pennsylvania (PA) and Washington (WA) State that are seeking to transform 
policies and practices to improve student outcomes through participation in Achieving the 
Dream. The study examined the early efforts of the 13 PA and WA colleges to implement the 
Achieving the Dream institutional improvement process. Its findings are based on extensive on- 
site interviews with personnel at all 13 colleges, a survey of data use by faculty and 
administrators at these colleges, and an analysis of data on the performance of the colleges in 
the period before they joined the initiative. The findings from this study will be compared with 
follow-on research that CCRC and MDRC plan to conduct in two years to see what progress the 
colleges have made by the end of the five-year project period. 



Overview of Achieving the Dream 

Traditionally, community colleges have played a vital role in American society by 
expanding access to a college education for millions of Americans. In recent years, community 
college educators, under pressure from government agencies, accreditation agencies, and 
students themselves, have begun to pay more attention to what happens to students once they 
enter college and to take steps to increase the rates at which community college students earn 
college credentials and transfer to baccalaureate institutions. 

One of the most important initiatives in this shift in community college attention from 
access to access and success is Achieving the Dream, a national initiative involving more than 
80 colleges in 15 states. The initiative seeks to help more community college students succeed 
and is particularly concerned about students of color and low-income students, who traditionally 
have faced significant barriers to success. ^ 



*For more information on Achieving the Dream, visit the initiative’s website at: 
www.achievingthedream.org. 
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The Achieving the Dream Culture of Evidence 
Principles and Process^ 

Most efforts to improve community college student success involve specific 
programmatic interventions. Achieving the Dream is based on the premise that to improve 
outcomes for students on a substantial scale colleges need to change how they do business in 
fundamental ways. Specifically, colleges should create a “culture of inquiry and evidence” in 
which decisions about the design, delivery, and funding of programs and services are made 
based on evidence of what works to improve student outcomes. Colleges that operate in this 
way adhere to four principles: 

• Committed leadership. The college’s senior leaders actively support efforts 
to improve student success, not just enrollments, and are committed to 
achieving equity in student outcomes across racial, ethnic, and income 
groups. Senior administrators, board members, and faculty and staff leaders 
demonstrate a willingness to make changes in policy, procedures, and 
resource allocation to improve student success. 

• Use of evidence to improve programs and services. The college has 
established processes for using data on student progression and outcomes to 
identify gaps in achievement, and to formulate strategies for addressing the 
gaps and evaluating the effectiveness of those strategies. 

• Broad engagement. Faculty, staff, administrators, and students share 
responsibility for student success and work together to assess the 
effectiveness of programs and services and make improvements. 

• Systemic institutional improvement. The college has an established planning 
process that relies on data to set goals for student success and measure goal 
attainment. Decisions about budget allocations are based on evidence of 
program effectiveness and are linked to plans to increase student success. 

The college offers faculty and staff professional development opportunities 
that reinforce efforts to improve student outcomes and close achievement 
gaps. 

Achieving the Dream recommends that colleges transform themselves according to 
these principles and thereby build a culture of evidence through a five-step process: 



^This section draws from the Achieving the Dream (2007) Framework for Improving Student Outcomes 
and Institutional Performance, which describes the initiative’s model for institutional effectiveness. Figure 1.1, 
which illustrates the process, is taken from a recent report by Elizabeth Zachry (2008) of MDRC. 
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Step 1: Commit to improving student outcomes. The college’s senior 
leadership, with support from the board of trustees and faculty leaders, 
commits to making the changes in policy and resource allocation necessary to 
improve student outcomes, communicates the vision widely within the 
college, and organizes teams to oversee the process. 

Step 2: Use data to identify and prioritize problems. The college uses 
longitudinal student cohort data and other evidence to identify gaps in student 
achievement. A key premise of this approach is that once faculty and staff see 
that certain groups of students are not doing as well as others they will be 
motivated to address barriers to student success. To ensure that they focus 
their resources to greatest effect, colleges are encouraged to prioritize the 
student achievement problems that they plan to address. 

Step 3: Engage stakeholders in developing strategies for addressing priority 
problems. The college engages faculty, staff, and other internal and external 
stakeholders in developing strategies for remedying priority problems with 
student achievement, based on a diagnosis of the causes and an evaluation of 
the effectiveness of previous attempts by the institution and others to address 
similar problems. 

Step 4: Implement, evaluate, and improve strategies. The college then 
implements the strategies for addressing priority problems, being sure to 
evaluate the outcomes and using the results to make further improvements. 

Step 5: Institutionalize continuous improvement of programs and services. 

The college takes steps to institutionalize processes for improving the impact 
of programs and services on student outcomes. Attention is given to how 
resources are allocated to bring new initiatives to scale and sustain proven 
strategies. Processes for program review, planning, and budgeting are driven 
by evidence of what works best for students. 

Achieving the Dream expects that by following this institutional transformation 
process, colleges will be able continuously improve rates of student success, including increased 
persistence, course pass rates, and, ultimately, credential attainment. Figure 1.1 illustrates the 
initiative’s theory of action. 
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Achieving the Dream: Community Colleges Count 



Figure 1,1 

Theory of Action for the Achieving the Dream Initiative 




Improved student 
outcomes and reduce 
achievement gaps 
between subgroups of 
students based on five 
indicators: 

• Completion of 
developmental 
courses and 
progression to credit- 
bearing courses 

• Completion of 
“gatekeeper" 
courses, particularly 
first college-level or 
degree-credit courses 
in math and English 

• Completion of 
attempted courses 
with a “C” or better 

• Persistence from 
term to term and 
year to year 

• Attainment of 
credentials 
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Achieving the Dream provides both financial and technical support to help colleges 
undertake this process. The financial support includes a one-year planning grant and 
implementation funding over four years that colleges can use to support data collection and 
analysis, engagement of faculty and staff, and implementation of improvement strategies. The 
technical support includes two outside consultants — a coach (usually a former community 
college president) and a data facilitator (usually a community college institutional researcher) 
— who advise the college on how to analyze its data on student success, interpret and 
communicate the findings to faculty and staff, and use the information to make improvements in 
college programs and services. The coach and data facilitator each spend 12 days working with 
the colleges during the planning phase and the first year of implementation, and then gradually 
reduce their time in subsequent years. In addition, teams from all of the colleges attend an 
annual institute designed to foster sharing of effective strategies. 



Achieving the Dream Colleges in Pennsylvania and Washington 

To date, more than 80 colleges in 15 states have joined Achieving the Dream and 
embarked on the institutional improvement process with financial support from Lumina 
Foundation for Education and other funders. 

This report examines the planning and initial implementation work at 13 colleges, 7 in 
Pennsylvania and 6 in Washington State, that joined Achieving the Dream in 2006 in the third 
round of entering colleges. Table 1.1 identifies these colleges and presents some salient 
characteristics of the students they serve. 



5 




Achieving the Dream: Community Colleges Count 
Table 1.1 



Achieving the Dream Colleges in Pennsylvania and Washington State 
Selected Characteristics, Academic Year 2005-06 



College 


Enrollment (FTE) 


Minority 
Enrollment (%) 


Pell Recipients 

(%) 


Pennsvlvania 


Allegheny County 


12,443 


28 


34 


Beaver County 


1,886 


20 


37 


Delaware County 


3,664 


29 


21 


Montgomery 


5,684 


31 


18 


Northampton 


4,525 


22 


23 


Philadelphia 


13,542 


68 


54 


Westmoreland 


4,116 


11 


40 


Washington State 


Big Bend 


1,464 


29 


44 


Highline 


4,635 


48 


18 


Renton Technical 


2,782 


51 


27 


Seattle Central 


4,912 


47 


21 


Tacoma 


5,064 


39 


30 


Yakima 


3,592 


38 


40 



SOURCE: U.S. Department of Education, National Center for Education Statistics, Integrated 
Postsecondary Education Data System (IPEDS). 

The participating colleges in the two states were chosen through somewhat different 
processes. In Pennsylvania, a request for proposals was sent to 14 community colleges that 
serve high percentages of low-income students or students of color based on IPEDS data. The 
percentage of students who receive Pell grants was used as a proxy (admittedly imperfect) of 
the proportion of students who are low income. All 14 Pennsylvania community colleges 
submitted proposals. Of them, 6 were chosen to receive one-year planning grants of $50,000 by 
reviewers organized by the national initiative. One other institution. Community College of 
Allegheny County (CCAC), was allowed to participate with its own funding. In Washington 
State, the 6 colleges that enrolled the highest proportions of low-income and/or minority 
students among the 34 community and technical colleges in the state were asked by the State 
Board for Community and Technical College (SBCTC) to submit proposals. All 6 submitted 
proposals and, following a review to ensure that the proposed activities were sound, all were 
awarded planning grants. 

The 13 participating PA and WA colleges were provided with travel funds for five 
“core team” members to participate in a Kickoff Institute in July 2006. The core team was to 
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include the college’s president, vice presidents or deans for academic affairs and student 
services, a faculty representative, and a person responsible for institutional research (IR) or 
effectiveness (IE). At the Kickoff Institute, each college’s core team met with its coach and data 
facilitator to scope out a plan for the planning year. Back on campus, the colleges were advised 
to organize a “data team” consisting of institutional researchers and others who would conduct 
the necessary data analyses to inform the core team as it examined data on student progression, 
designed strategies for increasing student success, and, by the end of the planning year, prepared 
a multi-year proposal for implementing the strategies. The coach and data facilitator visited their 
colleges and met with the core and data teams at each. 

At the end of the planning year, all 13 PA and WA colleges submitted proposals to 
implement the strategies for improving student success that they developed through the 
planning process. Six of the PA colleges received four-year implementation grants of $200,000 
($50,000 per year) from the Heinz Endowments. CCAC decided to continue to participate in the 
initiative using its own funds. All six Washington colleges received four-year implementation 
grants of $400,000 ($100,000 per year) from the Education Assistance Eoundation (now called 
College Spark Washington). Throughout the four-year implementation period, all participating 
colleges will continue to receive technical assistance from their coach and data facilitator as 
well as support to attend annual Strategy Institutes, where teams from all Achieving the Dream 
colleges meet to share promising practices. 

In effect, the goal of the planning year was to get the colleges started on the institutional 
transformation process by focusing on the first three steps: (1) commit to improving student 
outcomes; (2) use data to identify and prioritize problems, and (3) engage stakeholders in 
developing strategies for addressing priority problems. The four-year implementation plan 
developed during the planning year was designed to guide the colleges as they carried out the 
fourth step of the process: implement, evaluate, and improve strategies for improving student 
success. During the implementation period, colleges are expected to continue the first three 
steps of the process focused on further identifying gaps in student achievement and developing 
new strategies, and to begin the fifth step aimed at institutionalizing a culture of evidence on 
their campuses. 



Research Questions 

This report examines the efforts by the 13 Achieving the Dream colleges in 
Pennsylvania and Washington to begin implementing the initiative’s institutional improvement 
process during planning and first implementation year.^ 



^For the sake of brevity, we will refer to the Pennsylvania colleges as the “PA colleges” and the 
Washington State colleges as the “WA colleges.” 
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Specifically, the report addresses the following research questions: 

• How closely did the PA and WA colleges follow the planning process 
recommended by Achieving the Dream? What obstacles did they encounter? 

• Are any of the colleges using particularly innovative or effective methods for 
communicating the Achieving the Dream vision to stakeholders within and 
outside of the college and engaging faculty and student services staff on a 
wide-scale in the improvement process? 

• What student success strategies are the colleges implementing, how much 
progress have they made on implementation, and what have been the 
preliminary results? 

• To what extent is the Achieving the Dream work at these colleges focused on 
addressing achievement gaps and increasing equity in student outcomes 
across racial or ethnic and income groups? 

• In what ways are faculty and administrators at the PA and WA Achieving the 
Dream colleges using data on student outcomes? 

• How far along are these colleges in implementing the Achieving the Dream 
principles of institutional improvement and thereby building a culture of 
evidence? To what extent have colleges linked their work on Achieving the 
Dream with other efforts to bring about systemic improvements in 
institutional performance? 

• Has Achieving the Dream contributed to the colleges’ progress to date in 
building a culture of evidence for student success? What more can the 
initiative do? What more do the colleges themselves need to do? 

In addition, this study parallels baseline evaluation research that CCRC and MDRC 
conducted with the 27 colleges that comprised the Achieving the Dream cohort in the first 
round at a similar stage of their work (spring of the first implementation year)."^ Since the 
initiative has learned from the experience documented in that earlier report, this study sought to 
see if there is evidence that the third-round colleges in PA and WA have been able to make 
faster progress than the first-round colleges in building a culture of evidence at a similar stage in 
their participation in the initiative. 

This report presents a baseline analysis of the early efforts of the PA and WA colleges 
in implementing the Achieving the Dream improvement process. CCRC and MDRC plan to 



'‘Brock et al. (2007). 




conduct a second round of visits and data analysis in two years. We will use this baseline 
assessment to gauge the progress colleges have made by the end of the five-year project period. 



Methodology 

To address the questions presented above, CCRC and MDRC took a multi-pronged 
approach to the research. 

Field Research 

The research team visited all 13 colleges in spring 2008. At each institution, evaluators 
interviewed key personnel involved with the initiative, including college presidents, vice 
presidents, deans, institutional researchers, and faculty members. The interview protocol was 
based on the Achieving the Dream Framework for Improving Student Outcomes and 
Institutional Performance, which describes the initiative’s model for institutional effectiveness. 
Appendix A presents a tool that the research team used to gauge the extent to which colleges 
have implemented practices that reflect the various principles of this model. 

The interviews covered a range of topics, including how colleges organized and carried 
out the planning process; what strategies were identified; how broad the involvement of faculty, 
staff, and others was in the effort; and what impact, if any, the Achieving the Dream work by 
college personnel and outside support from the initiative had on colleges’ efforts to improve 
student outcomes. The evaluators also interviewed a few faculty members on each campus who 
were not directly involved in the initiative to gauge their awareness of Achieving the Dream and 
to ask for their perceptions about efforts to improve student outcomes at the college. Most 
interviews were conducted individually or in small groups and lasted about an hour. The 
interviews followed a protocol to ensure that similar questions were asked of comparable people 
at all of the colleges. The notes generated from these interviews were analyzed using the tool in 
Appendix A. To protect confidentiality, names of individual respondents or colleges are not 
identified in this report. 

Survey of Data Use by Faculty and Administrators 

CCRC and MDRC also conducted a survey of the use of student data by faculty and 
administrators at the PA and WA colleges as well as at the 27 first-round Achieving the Dream 
colleges. The survey asked full-time faculty and administrators about what student data they 
use, how accessible data on students are at their college, how they use data in their jobs, and 
what types of data they find most useful. It also asked respondents about their familiarity and 
involvement with Achieving the Dream. The survey, conducted over five months beginning in 
September 2007, received a very favorable response rate: 60 percent of faculty and 73 percent 



9 




of administrators surveyed responded. In this report, we compared the average responses to 
examine the patterns of data use in the PA and WA Achieving the Dream colleges to ascertain if 
there were notable differences between the PA and WA colleges, and between these third-round 
colleges and those that entered in the first round. 

Analysis of Baseline Data 

CCRC and MDRC also analyzed data on student progression and outcomes that the 
Achieving the Dream colleges were required to submit to a centralized database managed by the 
initiative. Specifically, we examined the performance of the PA and WA colleges on the five 
indicators established by the initiative for participating colleges. They include completion of 
developmental courses in math, English, and reading; completion of “gatekeeper” courses (that 
is, the first college-level courses) in English and math; the ratio of completed credits to 
attempted credits; persistence from semester to semester and year to year; and completion of 
certificates, diplomas, or associate degrees. We calculated average institutional rates on each 
indicator for all students and sub-groups defined by race/ethnicity, gender. Pell grant receipt (as 
a proxy for low-income status), and referral to developmental courses. We compared the 
performance on these measures of the PA and WA colleges for the three-year period before 
they joined the initiative as a baseline for examining their performance after they joined. We 
also compared the baseline performance of the 13 third-round colleges with that of the first- 
round colleges during a similar three-year period prior to joining Achieving the Dream. 

Comparison with Baseline Findings of First-Round Colleges 

The findings from this study of the PA and WA Achieving the Dream colleges were 
compared with those of the baseline evaluation of the colleges that joined the initiative in the 
first round. That evaluation was also conducted by CCRC and MDRC. 



Organization of the Report 

This report is organized as follows: Chapter 2 presents statistics on the baseline 
performance of the PA and WA colleges prior to joining the initiative using the data submitted 
by the colleges to the initiative’s national database. Chapter 3 presents results from the survey of 
patterns of data use by faculty and administrators at the PA and WA colleges compared with 
those in the first-round colleges. Chapter 4 examines the initial efforts by colleges on the first 
three steps of the Achieving the Dream institutional improvement process during the planning 
year. Chapter 5 describes the programmatic strategies for improving student success that the 
colleges identified during the planning year and are now implementing (as part of the fourth 
step of the improvement process) during the four-year implementation period that began in fall 
2007 before our spring 2008 visits. Chapter 6 assesses the initial progress of the colleges in 
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institutionalizing and sustaining the institutional improvement model (step 5 of the process) 
reflected in the four Achieving the Dream principles. Chapters 1 through 6 each includes a 
comparison with the first-round Achieving the Dream colleges. Chapter 7 concludes with an 
assessment of the extent to which Achieving the Dream has benefited the PA and WA colleges 
to date and makes recommendations for ways the colleges and the initiative might improve the 
impact of their work. 
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Chapter 2 



Baseline Performance of the 
Pennsylvania and Washington Colleges 

Introduction 

At the beginning of the initiative, the Achieving the Dream partners established five 
main performance indicators for participating colleges. The indicators are the rates at which a 
college’s students: 

(1) Successfully complete remedial or “developmental” courses and progress to 
credit-bearing courses. 

(2) Enroll in and successfully complete college-level “gatekeeper” courses. 

(3) Complete the courses they take, with a grade of C or higher. 

(4) Reenroll from one semester to the next. 

(5) Earn certificates and/or degrees. 

These indicators were chosen because most community colleges can readily measure 
them. Also, they reflect the importance of tracking community college students’ progress over 
time across intermediate milestones since community college students often take a long time to 
earn credentials. Moreover, a substantial number of degree-seeking community college students 
have to take developmental courses. Many of them do not progress to college-level coursework, 
and, of those who do, too many do not pass their first college-level courses. Specific measures 
were developed for the five performance indicators. They are identified in the tables in 
Appendix B. 

Colleges participating in Achieving the Dream are required to report unit record data on 
cohorts of first-time, degree-seeking students to a database maintained by the initiative. They 
submit data for new fall cohorts and regular updates on the progress of earlier cohorts. By the 
end of the four-year Achieving the Dream implementation phase, each PA and WA college will 
have submitted at least two years of data on six cohorts of students — three cohorts prior to 
implementation and three after. This will make it possible to compare rates and identify trends 
for students who enrolled before the start of the implementation phase and those who enrolled 
after. 



Achieving the Dream expects each college both to improve overall student outcomes on 
the indicators and to narrow the gaps in attainment among students groups. 
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This chapter presents statistics on the baseline performance of the PA and WA colleges 
on the Achieving the Dream student achievement measures for the three-year period before 
each college joined Achieving the Dream. It examines differences between the baseline 
performance of the PA colleges and the WA colleges and compares the performance of these 
third-round colleges with that of the first-round colleges. These baseline data will be used to 
identify trends among participating colleges by comparing the baseline rates with the rates of 
cohorts of students who enroll after the Achieving the Dream implementation phase. 

The statistics presented here are based on institutional means, not averages for the 
pooled sample of all students. Thus, each college is weighted equally, regardless of the size of 
its enrollment. 



Baseline Performance 

The average institutional rates for PA and WA colleges on most of the baseline 
performance measures were low. This finding was not unexpected given that Round 1 colleges 
also had low rates for their baseline measures. 

While there was variation in the average performance rates for WA, PA, and Round 1 
colleges on all of the Achieving the Dream measures, there was often more substantial variation 
within these three groups than among them. For example, average rates for successful 
completion of highest-level developmental math were 37 percent (PA), 27 percent (WA), and 
29 percent (Rd 1), while the range for PA colleges was much larger (51 percent to 7 percent) 
than the range for WA colleges (31 percent to 22 percent). 

PA had a higher average rate than WA on 10 measures of the indicators and WA had a 
higher average rate on the other 7. The difference between PA and WA was less than 1 
percentage point on 2 of the 17 measures, and the difference was greater than 10 percentage 
points on only 3 measures, indicating that overall the spread in average rates between the two 
states was not great on most measures. 

The following sections of this chapter present the statistics for average institutional rates 
on each of the 17 specific measures of the five Achieving the Dream performance indicators. 
The statistics are based on the progression and outcomes of cohorts of first-time community 
college students in fall 2004 who were tracked over three years. In addition to the information 
on the figures in each section, the tables in Appendix B provide more detailed statistics on each 
measure. 
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Developmental Courses 

The first set of measures shows the average rates at which students completed the 
highest-level developmental education courses at PA, WA, and Round 1 colleges (Figure 2.1). 
A significant percentage of degree-seeking community college students takes developmental 
courses and many do not progress to college-level coursework. Therefore, completion of the 
highest-level developmental courses is an important intermediate milestone and a key area for 
colleges’ improvement process. 

PA colleges had a higher average rate of successful completion for developmental 
instruction in all three subjects (math, English, and reading) than WA colleges had. WA rates 
were slightly lower than those in PA, although rates in WA did not vary as widely as in PA. In 
developmental math, for example, PA rates ranged from 7 percent to over 50 percent, whereas 
WA rates only ranged from 22 percent to 3 1 percent. 

Turning to a comparison with Round 1 colleges, both PA and WA colleges had higher 
average rates of completion for developmental English than Round 1 colleges, but Round 1 
colleges had a higher completion rate than both PA and WA in developmental reading. 
Interestingly, the lowest rate for Round 1 colleges among the three developmental subjects was 
in math, but for both PA and WA, the lowest rate among the three subjects was in reading. 
Taking into account all of the average rates for all three sets of data, the highest average 
completion percentage was PA students in developmental English (about 45 percent) and the 
lowest completion percentage was WA students in developmental reading (about 20 percent). 
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Achieving the Dream: Community Colleges Count 
Figure 2,1 

Successful Completion of the Highest Level Developmental Courses 
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Gatekeeper Courses 

Rates of completion of the first-level “gatekeeper” courses in math and English are 
important measures for Achieving the Dream because studies have shown that passing these 
courses is associated with a higher likelihood of graduating.^ Therefore, student success in 
gatekeeper courses is one of the important areas of improvement for Achieving the Dream 
colleges. 

PA and WA colleges had higher average rates of completion in gatekeeper English 
courses (46 percent and 42 percent, respectively) than they did in gatekeeper math courses (28 



^Calcagno, Crosta, Bailey, and Jenkins (2007). 
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percent and 27 percent). Interestingly, in both PA and WA, the average rates at which students 
completed gatekeeper English courses were higher for students who were referred to 
developmental instmction than for students who were not referred to remediation (Figures 2.2 to 
2.5, Table B.8). In PA, students referred to developmental instruction completed at about 49 
percent while those not referred completed at about 45 percent. The difference was more 
pronounced among WA colleges, where students referred to remediation completed gatekeeper 
English at 53 percent while those not referred to remediation completed at 46 percent. It may be 
that students who were referred to and successfully completed developmental English were 
better prepared for college-level English courses. It is not clear why this difference only 
occurred for gatekeeper English and not gatekeeper math. 

Both PA and WA colleges had higher average rates of completion in both math and 
English gatekeeper courses than did Round 1 colleges. 

Because of the correlation between success in gatekeeper courses and an increased 
likelihood of graduating, and the relatively low rates of successful completion of English and 
math gatekeeper courses among colleges in PA and WA, these colleges are advised to find 
ways to improve their performance on this critical benchmark. Additional analyses of the 
baseline data (and data for subsequent cohorts) may provide better insight into what is 
preventing successful completion of these courses and lead to the development of targeted 
strategies that address this challenge. Institutional improvements in this area may help to 
increase rates of certificate or degree completion. 
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Achieving the Dream: Community Colleges Count 
Figure 2,2 

Successful Completion of Gatekeeper Math Course 




Achieving the Dream: Community Colleges Count 
Figure 2,3 

Successful Completion of Gatekeeper English Course 
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Achieving the Dream: Community Colleges Count 
Figure 2,4 



Successful Completion of Gatekeeper Math Course by Referral Status 
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Achieving the Dream: Community Colleges Count 
Figure 2,5 

Successful Completion of Gatekeeper English Course by Referral Status 




Course Completion 

The average course completion rates for PA and WA colleges were very similar (76 
percent and 79 percent, respectively). However, PA colleges ranged from 59 percent to almost 
94 percent, a much larger variation than WA colleges, which ranged from 75 percent to 88 
percent (Figure 2.6). The difference in range suggests that students across the WA colleges were 
more consistently completing courses. The course completion rate at Round 1 colleges did not 
differ dramatically from PA and WA, although it was slightly lower at 70 percent. Round 1 
colleges exhibited a range similar to PA (52 percent to 92 percent). 
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Achieving the Dream: Community Colleges Count 

Figure 2,6 

Ratio of Completed Credits to Attempted Credits 
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Persistence 

As would be expected, the average rates of enrollment decreased as the period of time 
from initial enrollment increased (Figures 2.7 to 2.9). WA colleges had the highest percentage 
of students persisting across the three measured periods of time.^ Interestingly, as time passed, 
the gap between WA colleges’ rates of persistence and both PA and Round 1 colleges’ rates of 
persistence increased. All three sets of colleges showed persistence rates between 70 and 74 
percent in the first semester after the initial term of enrollment. By the third year persistence 
measure, however, the gap between WA colleges (40 percent) and PA (30 percent) and Round 
1 colleges (33 percent) had widened. This increased gap may help to explain why WA colleges 
had a higher average rate of completion within three years (27 percent) than both PA and Round 
1 colleges (both about 10 percent). Across the three-year period measured, PA and Round 1 
colleges exhibited very similar average rates of persistence (Figure 2.10). 

The decrease in student persistence over time is not surprising given the low rate at 
which students earned a certificate or degree within three years at the colleges. With an average 



^For the three measures of persistence included in the baseline performance of PA and WA colleges, the 
average rates of persistence took into account any students who completed within the years of the measures. 
This approach was taken to ensure that the colleges’ rates of persistence would not be negatively affected by 
including in the number of students not persisting in subsequent semesters those students who did complete. 
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completion rate of just over 10 percent, PA colleges in particular need to focus on the decline in 
enrollment in semesters after students initially enroll. Further analyses of student data to inform 
the development of strategies focusing on increasing student persistence, and specifically 
emphasizing continuous enrollment from one semester to the next, will likely help to increase 
the rate at which students earn college credentials. 



Achieving the Dream: Community Colleges Count 
Figure 2,7 

Enrolled in the First Semester after the Initial Term or Completed Within One Year 
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Achieving the Dream: Community Colleges Count 
Figure 2,8 

Enrolled in at Least One Semester in the Second Year 
or Completed Within Two Years 
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Achieving the Dream: Community Colleges Count 
Figure 2,9 

Enrolled in at Least One Semester in Each of the First Three Years 
or Completed Within Three Years 
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Achieving the Dream: Community Colleges Count 
Figure 2.10 
Persistence 
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Credential Completion 

PA colleges’ average rate of credential completion closely matched the Round 1 
colleges at about 10 percent, while WA’s average rate was higher, with a credential completion 
rate of just over 27 percent. However, when measuring the rates of students enrolled in at least 
one semester in the t hird year, WA colleges had a lower rate (21 percent) than both PA colleges 
(31 percent) and Round 1 colleges (29 percent). This difference suggests that students in WA 
colleges were more successful at completing credentials programs in three years, while PA and 
Round 1 colleges had higher percentages of students still enrolled during the third year (Figures 
2.11 to 2.15). 

WA also had higher rates of obtaining an associate degree within three years (16 
percent to 9 percent for PA and 7 percent for Round 1) and of obtaining a certificate or diploma 
within three years (11 percent to 1.5 percent for PA and 3.5 percent for Round 1 colleges). In 
addition to a rate of 1 1 percent in WA, there was also a significantly greater range of rates in 
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WA for obtaining a certificate or diploma within three years, from under 2 percent to 51 
percent. WA’s rates may have been higher because of a greater emphasis on career technical 
programs, which include occupational certificates, in the state. Similarly, the considerable range 
in the rates at which students in WA obtained a certificate or diploma may be the result of 
certain colleges in the state emphasizing certificate attainment in career and technical programs 
while others emphasize academic degrees and transfer. It could also mean that the colleges with 
higher rates on these measures are serving their students more effectively. 



Achieving the Dream: Community Colleges Count 
Figure 2,11 

Completed Within Three Years 
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Achieving the Dream: Community Colleges Count 
Figure 2,12 

Obtained an Associate Degree Within Three Years 




Achieving the Dream: Community Colleges Count 
Figure 2,13 

Obtained a Certificate or Diploma Within Three Years 




Achieving the Dream: Community Colleges Count 

Figure 2,14 

Enrolled in at Least One Semester in the Third Year 
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Achieving the Dream: Community Colleges Count 

Figure 2.15 

Completion Rates for PA, WA, and Round 1 Colleges 
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Pell Recipient Status 

The federal Pell Grant program provides need-based grants to low-income students to 
promote access to postsecondary education. As such, Achieving the Dream is using the 
percentage of students who receive Pell Grants as a proxy measure for the proportion of 
students who are low income. 

WA exhibited higher average rates of completion within three years for both Pell 
recipients and nonrecipients than did PA and Round 1 colleges. There was much less variation 
among PA colleges than WA colleges. Round 1 and PA colleges exhibited comparable average 
rates for both recipients and nonrecipients (Figures 2.16 and 2.17). 

Consistent with Round 1 colleges, rates of persistence for WA and PA colleges were 
higher for Pell recipients than nonrecipients. However, for all three sets of data there was no 
significant difference in completion rates between Pell recipients and nonrecipients. In PA, for 
example, 66 percent of Pell recipients persisted into the second year but only 12 percent 
completed within three years. Similarly, 46 percent of nonrecipients persisted but only 10 
percent completed within three years. These findings suggest that while Pell recipients may 
have an advantage over nonrecipients on intermediate measures of progress, such as 
persistence, this advantage does not appear to carry over to success in completing credentials. 



Achieving the Dream: Community Colleges Count 
Figure 2,16 

Completion Within Three Years of Pell Recipients and Nonrecipients 
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Achieving the Dream: Community Colleges Count 
Figure 2,17 

Comparison of Persistence and Completion by Pell Status 
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Race and Ethnicity 

Achieving the Dream is particularly concerned about student groups that have faced the 
greatest barriers to success in college. Colleges are expected to identify and work toward 
closing any substantial gaps in performance on the five indicators among these groups of 
students, such as racial or ethnic minorities. Table B.4 shows the average institutional rates for 
the 17 performance measures as analyzed by race and ethnicity for PA and WA colleges. 

The average institutional rates for successful completion of developmental courses and 
gatekeeper courses were lower on many of the measures for African-Americans, Hispanics, and 
Native Americans than for whites. PA colleges had more gaps on these measures than WA 
colleges, and in both states the completion rates for developmental and gatekeeper math had the 
most differences across race and ethnicity when compared with whites. In WA, whites and 
African-Americans had similar rates of completion for developmental English and reading. In 
these same two developmental subject areas, Hispanics in WA had higher rates of completion 
(50 percent for English and 25 percent for reading) than whites (37 percent for English and 21 
percent for reading). 
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In PA, all of the minority groups had lower average rates than whites for completion of 
gatekeeper math and English courses. While the same was true of gatekeeper math in WA (with 
the exception of Asians), a major difference between the two states was completion of 
gatekeeper English. In WA, Asians, African-Americans, Hispanics, and Native Americans all 
had higher rates of completion in gatekeeper English than whites. 

Across both PA and WA colleges, the rates of successful course completion were lower 
for African-Americans, Hispanics, and Native Americans than for whites — a gap also present 
among Round 1 colleges. There were also gaps in average rates for the measures of persistence 
in both PA and WA. Hence, course completion and persistence for minority students are areas 
where colleges in both states need to focus their efforts in identifying barriers and closing 
achievement gaps. By addressing disparities in attainment among particular student groups, 
colleges will likely see institution-wide improvements in student success. 

Average completion rates were highest in WA for minority students, whereas PA and 
Round 1 colleges exhibited similar, lower average rates (Eigure 2.18). While WA had the 
highest minority student completion rates, it also had the greatest achievement gaps when 
comparing rates for Hispanics (22 percent) and African-Americans (15 percent) with whites (30 
percent). WA colleges also exhibited the greatest range of completion rates for whites, 
Hispanics, and African-Americans. 
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Achieving the Dream: Community Colleges Count 
Figure 2,18 

Completion Within Three Years for Selected Race/Ethnic Groups 




Much has been written about low levels of educational attainment among African- 
American and Hispanic men.’ The completion rates for both PA and WA colleges showed 
these patterns of differences, with African-American and Hispanic males completing within 
three years at average rates that are lower than black and Hispanic females and black and 
Hispanic students generally. Tables B.5 and B.6 show institutional averages of the Achieving 
the Dream performance measures for females and males as analyzed by race and ethnicity. It is 
also interesting to note that comparisons between white students by gender and African- 
American and Hispanic students by gender for both PA and WA on average rates of completion 
show that there were greater differences between comparisons of the female groups than the 
male groups, with the exception of Hispanic females. The largest of these differences occurred 
among African-American and Hispanic women in WA. White women completed within three 
years at an average rate of 32 percent, while African-American and Hispanic women both 
completed at an average rate of about 16 percent (Figure 2.19). This 16 percent difference is 
higher than the difference for both African-American and Hispanic males when compared with 

’See, for example, Ashbum (2006); Cameron and Heckman (2001). 
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white males in WA. These patterns suggest that colleges need to continue to closely analyze 
their disaggregated data to identify appropriate strategies for closing the gaps based on student 
characteristics. 



Achieving the Dream: Community Colleges Count 
Figure 2,19 

Completion Within Three Years by Gender for Selected Race/Ethnic Groups 
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Summary 

Despite differences in the structure and oversight of the community college systems in 
PA and WA, the Achieving the Dream colleges from the two states exhibited similar average 
rates of performance on the Achieving the Dream measures prior to joining the initiative. The 
measures show that, on average, many students at these colleges are struggling academically. 
As the figures in this chapter illustrate, there was often greater variation among colleges within 
each state on many of the measures than there was when comparing the average rates of the two 
states. This difference indicates that individual colleges are entering the initiative with varying 
levels of student success. The baseline data alone do not provide enough information to 
determine the extent to which the variation among colleges is due to differences in institutional 
performance or differences in the readiness of the students served. 
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The average institutional rates for PA and WA colleges were generally low in all 
measures prior to the start of the initiative. It is expected that colleges that are successful in 
implementing large-scale student success strategies under Achieving the Dream will see 
improvements on the performance indicators. Such colleges should also be able to ameliorate 
the often substantial gaps in performance among minority students compared with whites. 

Overall, the baseline performance of the PA and WA colleges was similar to that of the 
Round 1 colleges. Therefore, it may be possible to determine if PA and WA colleges are able to 
make faster progress than the Round 1 colleges in improving student outcomes. 
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Chapter 3 

Patterns of Data Use by Faculty 



Introduction 

This chapter presents findings from the survey that CCRC and MDRC conducted in late 
2008 on the use of data by faculty and administrators at all the Achieving the Dream colleges.^ 
In particular, it summarizes key findings from the survey on patterns of data use in the 
Achieving the Dream colleges in Pennsylvania and Washington. The focus here is on data use 
by faculty, given the central interest of Achieving the Dream in engaging faculty in using 
evidence to improve teaching, although we also report responses to questions asked of 
administrators about use of data in their colleges generally. We were particularly interested in 
the extent to which faculty examine and use data on student progression and outcomes, such as 
developmental course completion rates and rates of persistence and graduation. Also of interest 
was how frequently faculty participated in organized discussions with other faculty on strategies 
for improving student success. 

In addition to presenting descriptive statistics, we report the results of analyses that 
compare the responses of the PA and WA colleges with each other and with those of the 
colleges that joined the initiative in the first round. ^ We hypothesized that because the first- 
round colleges had been involved in Achieving the Dream for two more years than the PA and 
WA colleges at the time of the survey, the extent to which faculty in the Round 1 colleges used 
data would be greater than that of faculty at the PA and WA colleges. 



Extent of Use of Data by Faculty 

Frequency of Data Use by Type 

Table 3.1 shows the percentages of faculty at the PA, WA, and Round 1 colleges, 
respectively, who indicated using or reviewing particular sorts of information at least once a 



*A more extensive discussion of the survey findings is presented in a report by CCRC and MDRC; see 
Jenkins and Kerrigan (2009). 

^The detailed results of these differences in means tests are not presented in this report. Given that the 
responses to the survey questions examined here were not normally distributed, we used the Mann-Whitney- 
Wilcoxon rank sum test instead of standard difference in means tests to identify those items on which faculty in 
either the PA or WA or Round 1 colleges or the Round 1 or Round 3 (PA and WA) colleges scored 
consistently higher than the comparison group. Given that we were conducting tests for large number of items, 
we used the Bonferroni adjusted level of significance and a conservative p-value (p < .001) to measure 
statistical differences. 
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year, as well as the percentage who said that they never use a given type of information. Not 
surprisingly, high percentages of faculty at the three groups of colleges reviewed or used grades 
and course evaluations at least once a year. An unexpected finding is the fact that at least once a 
year more than half the faculty at all three groups used data on placement test scores, retention 
rates, and graduation rates. More than half the faculty at WA and Round 1 colleges, and nearly 
half at PA colleges, used measures of student learning other than grades at least annually. Still, 
over a third of faculty never used such measures, and nearly a third never reviewed data on 
student achievement gaps among different student groups. Over 40 percent used information 
broken down by students’ race or ethnicity at least once a year, although a smaller percentage of 
faculty used data broken down by student income levels or receipt of financial aid at least once 
a year. This is not surprising because colleges generally do not have a reliable way to collect 
income data for all of their students, other than those who apply for financial aid. 
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Achieving the Dream: Community Colleges Count 

Table 3.1 



Frequency of Faculty Members’ Use or Review of Various Data Types 



Percentage of Faculty 
Using the Given Information Type* 



Type of Information 


At Least Once per Year 


Never 




PA 


WA 


Round 1 


PA 


WA 


Round 1 


Placement test scores 


69.1 


73.6 


68.2 


24.1 


20.9 


24.5 


Enrollment data 


84.0 


90.7 


87.6 


10.9 


6.8 


8.6 


Grades 


91.0 


86.7 


90.3 


6.0 


9.2 


7.0 


Course evaluations 


81.7 


89.6 


91.9 


7.2 


3.4 


5.1 


Measures of student learning 
other than grades 


47.7 


54.6 


57.0 


40.2 


34.4 


32.4 


Retention rates 


63.4 


69.0 


75.7 


21.9 


19.1 


15.6 


Graduation rates 


61.1 


57.4 


67.4 


23.5 


27.2 


21.4 


Transfer rates 
Percentage of students 


49.0 


41.0 


46.8 


32.3 


36.8 


36.4 


successfully completing 
developmental education 


44.5 


42.2 


50.2 


39.0 


42.5 


36.3 


Einancial aid 


19.6 


37.2 


40.0 


65.6 


49.7 


48.1 


College budget and finances 


36.3 


56.8 


54.8 


47.0 


30.0 


32.1 


Results from external surveys 


36.1 


30.6 


46.5 


43.4 


47.9 


35.5 


Eocus groups or other qualitative 
data 


41.1 


37.8 


40.3 


36.4 


34.2 


38.1 


Research by the college 


63.6 


49.5 


58.4 


17.5 


24.7 


22.0 


Outside research on effective 
practices 


71.2 


70.1 


68.0 


14.9 


13.8 


17.6 


Data on student achievement 
gaps 


49.7 


54.2 


50.2 


29.1 


30.9 


31.4 


Information broken down by 
students’ race or ethnicity 
Information broken down by 


40.6 


53.5 


47.4 


41.0 


27.4 


35.9 


students’ income levels or 
receipt of financial aid 


23.6 


35.0 


34.5 


59.1 


46.3 


50.6 



* “Type of Information Not Available” responses were treated as missing. 

NOTE: Shaded rows indicate the types of data whose use is promoted by Achieving the Dream. 
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Analyzing the differences between the responses to these questions (not reported here), 
the Jenkins and Kerrigan report found that the WA colleges scored significantly higher than the 
PA colleges in the frequency with which faculty used data broken down by students’ race or 
ethnicity and data disaggregated by students’ income level. The finding regarding race and 
ethnicity may stem from the fact that some of the PA colleges had few minority students. 
Faculty at Round 1 colleges were significantly more likely that those at the PA and WA 
colleges to indicate that they used data on retention and graduation rates frequently. The report 
found a similar pattern in the broader analysis of the survey results where we compared the 
means across the PA, WA, and Round 1 colleges of composite measures of the use of data by 
faculty and administrators. As was pointed out in report, this finding is consistent with the 
hypothesis that colleges that had been involved in Achieving the Dream longer would be more 
advanced in their use of data for improving student success. However, the findings are merely 
suggestive; they cannot be seen as definitive evidence of a causal relationship between 
Achieving the Dream and more extensive use of data for improvement. 

Perceived Usefulness of Data by Type 

A majority of faculty at the PA, WA, and Round 1 colleges found most of the types of 
information presented in Table 3.2 at least somewhat useful in their jobs. This is true even for 
data on the percentage of students successfully completing developmental education, a 
surprising finding given that we surveyed faculty across disciplines, not just in developmental 
programs. Over two thirds of faculty also found data on student achievement gaps useful. 
Achieving the Dream may have helped to increase awareness and use of this information 
among faculty, since faculty and administrators who participated in Achieving the Dream 
activities at their colleges were, not surprisingly, more likely to use data on student outcomes. 
However, these findings simply show correlation, not causation, so we cannot definitively 
attribute these patterns to Achieving the Dream. 

Two thirds of the faculty at the PA and Round 1 colleges (and nearly as high a 
percentage at WA colleges) indicated that research reports and other information that their 
college provided were generally helpful to their work as teachers. Nearly 80 percent of the 
faculty at PA, WA, and Round 1 colleges found outside research on effective practices useful in 
their roles as teachers. 
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Achieving the Dream: Community Colleges Count 

Table 3.2 

Faculty Members’ Perception of the Usefulness of Various Types of Information 

to Their Job 



Type of Information 


Percentage of Faculty Members Indicating 
Type of Info is “Somewhat Useful” to “Very 
Useful”* 




PA 


WA 


Round 1 


Placement test scores 


71.4 


74.2 


75.0 


Enrollment data 


68.1 


75.1 


77.3 


Grades 


83.0 


79.5 


86.5 


Course evaluations 


88.8 


91.1 


89.9 


Measures of student learning other than grades 


69.8 


71.9 


70.9 


Retention rates 


76.0 


80.3 


82.2 


Graduation rates 


65.9 


65.9 


73.3 


Transfer rates 


65.8 


58.8 


66.4 


Percentage of students successfully completing 
developmental education 


60.4 


59.0 


65.9 


Financial aid 


29.0 


43.1 


45.7 


College budget and finances 


36.1 


48.6 


55.3 


Results from external surveys 


52.3 


43.8 


57.0 


Focus groups or other qualitative data 


62.2 


60.2 


59.9 


Research by the college 


66.6 


60.2 


68.3 


Outside research on effective practices 


79.5 


79.6 


78.8 


Data on student achievement gaps 


69.2 


67.3 


69.9 


Information broken down by students’ race or 
ethnicity 


52.3 


58.1 


48.5 


Information broken down by students’ income levels 


43.2 


49.2 


45.9 


or receipt of financial aid 



* “Not Applicable” responses were treated as missing. 



There were few differences on average in the extent to which faculty at the PA, WA, 
and Round 1 colleges valued the various types of information in Table 3.2. Faculty at PA 
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colleges found information on financial aid and college budgets and finances less useful than 
did those at WA and Round 1 colleges, but that is probably because PA faculty generally used 
such data less than did WA and Round 1 faculty (Table 3.1). Faculty at Round 1 colleges were 
more likely to indicate that they found the results of external surveys such as the Community 
College Survey of Student Engagement (CCSSE) more useful in their roles as teachers. 

Use of Data and Research by Faculty in Teaching-Related Decisions 

As is evident from Table 3.3, the majority of faculty surveyed used data and research at 
least to some extent in decisions related to teaching. Around one in five indicated that they were 
heavy users of data and research for teaching decisions. A smaller percentage said that they 
used data and research “not at all” in teaching-related decisions. 



Achieving the Dream: Community Colleges Count 
Table 3,3 

Extent of Use by Faculty of Data and Research on Students for 
Teaching-Related Decisions 



Percentage of Faculty Who Used Data and Research 
for the Given Decision Type 

Decision Type 1 1 



At Least Some 


A Lot 


Not at All 




PA 


WA 


R1 


PA 


WA 


R1 


PA 


WA 


R1 


Curriculum 


73.5 


77.8 


76.7 


18.6 


18.0 


18.2 




12.4 


13.8 


Teaching practices 


79.9 


83.0 


82.8 


25.3 


21.1 


24.6 


■B 


9.3 


8.5 


Advising students 


79.0 


79.1 


81.7 


21.8 


19.6 


24.7 


12.1 


12.14 


10.8 


Identifying students 
who are struggling 
academically 


76.9 


74.2 


79.7 


22.1 


20.9 


22.7 


11.1 


13.4 


10.7 



There were no statistically significant differences in the responses to these questions 
between the PA and WA and Round 3 and Round 1 colleges. 
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Participation in Organized Discussions on improving Student Success 

Three quarters or more of faculty at the PA, WA, and Round 1 colleges indicated that 
they participated at least once a year in organized discussions on improving students’ academic 
achievement or on closing achievement gaps (Table 3.4). Somewhat smaller percentages — but 
still majorities — reported participating in discussions about the needs or performance of 
students of color or of low-income students in particular. 



Achieving the Dream: Community Colleges Count 
Table 3.4 

Frequency of Participation by Faculty Members in Organized Discussions at the 
College on Topics Related to Improving Student Success 



Percentage of Faculty Participating in Discussions on the 



Given Topic 

f^icr>iicci/\n 


At Least Once per Year 


Never 


PA WA R1 


PA WA R1 


Improving academic achievement or „ „„ „ 

, . , . ol .D /4. / II. ! 

closing achievement gaps 

Academic needs or performance of ^ ^ 

^ ^ ^ , jo.o o4.j 

students or color 

Academic needs or performance of , 

1 . 1 ^ OD.l oU.o JJ.O 

low-income students 


7.1 10.6 10.0 

23.0 18.6 29.7 

13.4 22.9 26.3 



Faculty at the PA and WA colleges were significantly more likely than those at Round 
1 colleges to indicate that they participated frequently in organized discussions about improving 
the academic performance of students of color. This difference might reflect the fact that at the 
time of the survey, the PA and WA colleges had recently completed the Achieving the Dream 
planning year, when colleges were strongly encouraged to examine gaps in achievement among 
students grouped by race and ethnicity and other characteristics. Still, the differences between 
the colleges by round were fairly small. 

Use of Data by Academic Departments 

Most faculty responding to the survey indicated that they were in departments that used 
data and research for programmatic decisions at least to some extent (Table 3.5). Approximately 
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one fourth were in departments that were heavy users of data to make program decisions. Only 
a small percentage of the faculty respondents were in departments that did not make use of data 
and research for such decisions. The pattern of responses to these questions was similar among 
the PA, WA, and Round 1 colleges. 



Achieving the Dream: Community Colleges Count 

Table 3,5 

Extent of Use by Faculty Members of Data and Research on Students by Department 
in Decision Making About Selected Issues 



Percentage of Faculty Whose Department Uses Data and Research for 
Decisions on the Given Topic 

jggygg (Question asked of faculty only)* 



At Least Some 


A lot 


Not at All 




PA 


WA 


R1 


PA 


WA 


R1 


PA 


WA 


R1 


Curriculum 


75.2 


77.0 


81.0 


23.3 


21.1 


27.0 


13.4 


10.7 


10.5 


Teaching practices 


78.2 


80.8 


82.8 


19.0 


21.2 


24.3 


11.8 


9.6 


8.9 


Tutoring or other 
academic support 


76.9 


76.2 


81.0 


21.1 


17.1 


24.0 


13.1 


10.1 


9.6 


Program planning 


78.8 


83.0 


84.4 


23.9 


24.9 


26.2 


9.3 


7.8 


7.1 


Academic program 
review or evaluation 


80.3 


83.5 


86.2 


25.4 


22.7 


28.5 


8.0 


8.7 


6.1 


Long-term strategic 
planning 


75.1 


78.5 


82.6 


18.8 


18.7 


24.5 


12.2 


8.9 


7.7 


Budgeting or resource 
allocation 


58.9 


66.7 


75.8 


12.2 


14.2 


20.1 


23.2 


13.5 


12.3 


Identifying and 




















redesigning high- 
failure-rate courses 


63.4 


65.0 


74.9 


18.1 


18.1 


23.3 


19.7 


19.2 


12.8 



* “Don’t Know” responses treated as missing. 



The broader analysis of the survey results from the PA, WA, and Round 1 colleges by 
Jenkins and Kerrigan found that the frequency with which faculty members used data in 
decision making varied by department. Faculty in general education were on average 
significantly less likely than faculty in other program areas to use data on student outcomes and 
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to use data and research in decisions related to their teaching on a frequent basis. In contrast, 
faculty who taught in developmental or for-credit occupational programs were more likely than 
those in other fields to do so. Developmental faculty members were also significantly more 
likely to participate in organized discussions on student achievement and to use data 
disaggregated by race, ethnicity, or income. Adult basic education faculty used data no more 
frequently than faculty in other areas. Interestingly, even though they were more likely than 
faculty in other areas to use data in teaching-related decisions, faculty in for-credit occupational 
programs were less likely to participate in organized discussions about student achievement or 
to use data broken down by race, ethnicity, or income. 



Accessibility of Data and Perceived Barriers to Data Use 

Sources of Data on Students 

Faculty at the Achieving the Dream colleges indicated that they used a variety of 
sources or methods to get information on groups of students (Table 3.6). Faculty at WA 
colleges were significantly less likely than those at PA and Round 1 colleges to do searches 
themselves using their college’s student information system or their college’s website or fact 
book. This difference is likely due to the problems that the WA community and technical 
colleges had in retrieving data from the legacy information system they shared. Efforts to 
upgrade or replace that system have been going on for several years now and have not yet been 
completed. About a th ir d of faculty at the PA, WA, and Round 1 colleges indicated that they 
generally did not need information about groups of students. 
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Achieving the Dream: Community Colleges Count 
Table 3,6 



Sources of Information Used by Faculty on Groups of Students 



Source 


Percentage of Faculty Indicating 
That They Used the Given Source* 




PA 


WA 


Round 1 


Searches using the college’s student information system 


34.5 


14.5 


37.0 


Data from the college’s website or fact book 


28.6 


21.2 


32.9 


Reports distributed by the college’s institutional research 
(IR) office or other departments 


32.4 


32.4 


32.9 


Requests to the IR or information technology (IT) staff 


34.3 


33.2 


31.0 


My department’s database 


20.7 


29.3 


24.0 


State databases or research reports 


8.7 


11.1 


9.6 


I generally do not need information about groups of 
students 


33.1 


33.4 


32.2 



* Respondents were asked to select “all that apply.” 



Accessibility and Quality of Information and Research 

A majority of faculty at the PA, WA, and Round 1 colleges indicated that they were 
able to access information they needed in a timely manner and that the information they 
received was accurate, although faculty from the WA colleges were less satisfied with their 
access to data (Table 3.7). WA college faculty were also significantly less satisfied than faculty 
at the PA and Round 1 colleges that the reports they received from the college were clear and 
easy to follow, and were provided in a timely fashion. Faculty at about half of the PA and 
Round 1 colleges indicated that their college’s institutional research function was adequately 
staffed to meet the demand for information, compared with a third of WA college faculty. Our 
fieldwork in WA indicated that at least some colleges were having trouble recruiting qualified 
IR staff for the salaries that the colleges were able to offer. PA college faculty were significantly 
more likely than those in WA and Round 1 colleges to indicate that their college’s institutional 
research staff was responsive to requests for information. More than half of the PA and Round 1 
faculty, and 44 percent of the WA faculty, indicated that the research reports and other 
information their college provided were generally helpful to their work as teachers. 
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Achieving the Dream: Community Colleges Count 

Table 3.7 



Perceptions Among Faculty Members 
About the Accuracy and Availability of Data and Research 



Percentage of Faculty Indicating They 



Perception 


“Agree” 


to “Strongly Agree” 






PA 


WA Round 1 


The data in the college’s student information system 
are generally accurate and error free. 


70.2 


57.8 


63.3 


The data I need are generally available in a user- 
friendly format. 


58.7 


47.3 


60.3 


The college’ s institutional research staff is responsive 
to requests for information. 


80.4 


64.0 


68.1 


The college’s institutional research staff is adequately 








staffed for the college’s information and research 
needs. 


49.9 


33.1 


53.0 


The reports and other information the college provides 








to administrators and faculty are typically clear and 
easy to follow. 


66.9 


53.7 


63.7 


I am able to obtain the information I need in a timely 
fashion. 


69.8 


50.7 


62.8 


The research reports and other information the college 








provides to faculty are generally helpful to our work 
as teachers. 


52.3 


44.7 


53.5 



Perceived Barriers to Use 

As shown in Table 3.8, around a th ir d of the faculty at the PA, WA, and Round 1 
colleges indicated that one reason that they did not use data and research was that they were too 
busy with their teaching responsibilities. Other than that, most faculty members indicated that 
using data and research on students was part of their responsibility as faculty and that they had 
the skills needed to analyze data. Between 20 and 27 percent of faculty said that the data 
available were not relevant to their jobs. The responses of faculty on these items were similar 
across the PA, WA, and Round 1 colleges. 
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Achieving the Dream: Community Colleges Count 
Table 3,8 



Reasons Given by Faculty for not Using Data and Research on Students 



Percentage of Faculty Indicating They 



Reason 


“Agree” 


to “Strongly Agree” 






PA 


WA Round 1 


I am too busy with my teaching responsibilities. 


30.4 


37.9 


31.0 


It is not part of my responsibilities as a faculty 
member/administrator. 


14.3 


10.4 


13.8 


I do not have the research skills to understand and use 
data and research. 


13.4 


16.6 


17.5 


I do not trust the data that are available. 


12.9 


18.0 


16.9 


The data that are available are not relevant to my role 
as a faculty member/administrator. 


21.2 


27.0 


20.1 



Training for Data Use 

The percentage of faculty who indicated that they had been involved in training or 
professional development on institutional research or data analysis in the past year ranged from 
28 percent for the WA college faculty to 39 percent for the Round 1 college faculty (Table 3.9). 
The difference in responses between the Round 3 (PA and WA) and Round 1 colleges is 
statistically significant. Over half of the faculty at the PA, WA, and Round 1 colleges said that 
they participated in training or professional development on program evaluation or assessment. 
The broader analysis of the survey data in the Jenkins and Kerrigan report found, not 
surprisingly, that faculty who had recently participated in training or professional development 
in either of these topics were more likely to use data in their work. However, as was pointed out 
in that report, this finding does not necessarily mean that colleges could increase data use by 
increasing the amount of training provided, since it is possible that faculty and administrators 
who were heavier users of data were more likely to seek out training in data use. 



46 







Achieving the Dream: Community Colleges Count 
Table 3.9 



Involvement by Faculty in Training or Other Professional Development 

in the Past Year 



Percentage of Faculty Indicating That They 
Topic Participated in the Given Training in the 







Past Year 






PA 


WA 


Round 1 


Institutional research and/or data analysis 


30.5 


28.1 


38.7 


Program evaluation and/or assessment 


60.9 


62.4 


55.7 



Use of Data in Decision Making 

The survey asked administrators at the PA, WA, and Round 1 colleges to assess how 
much their college used data and research on students in decision making. As is evident from 
Table 3.10, the majority of respondents indicated that their college used data and research on 
students in decision making on program and planning issues at least to some extent. A th ir d or 
more indicated that their college used data and research extensively. Only a small fraction 
indicated that their college did not use data and research in decision making. There were no 
significant differences in the ranking of responses from administrators across the PA, WA, and 
Round 1 colleges. 
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Achieving the Dream: Community Colleges Count 
Table 3.10 

Extent of College’s Use of Data and Research on Students in Decision Making 



Percentage of Administrators Indicating Use of Data and Research by 
Their College for Decisions on the Given Topic* 



At Least Some 


A Lot 


Not at All 




PA 


WA 


R1 


PA 


WA 


R1 


PA 


WA 


R1 


Curriculum 


91.1 


85.7 


86.7 


37.6 


21.1 


30.0 


4.1 


2.3 


5.1 


Program planning 


91.4 


90.5 


88.7 


37.1 


25.9 


31.7 


3.0 


0.7 


4.6 


Academic program 
review or evaluation 


91.7 


87.5 


89.2 


41.2 


31.3 


34.6 


3.0 


0.7 


3.9 


Long-term strategic 
planning 


92.5 


88.2 


88.3 


44.6 


29.4 


36.6 


3.2 


0.7 


3.8 


Budgeting and 
resource allocation 


89.0 


86.5 


86.2 


40.3 


29.7 


34.9 


4.2 


0.7 


5.1 


Identifying areas for 




















improvement at the 
college 


90.7 


89.4 


89.4 


41.3 


32.5 


37.5 


2.7 


0.6 


3.2 



* Question asked of administrators only. 



In a question not reflected in Table 3.10, the vast majority of administrators across the 
three sets of colleges (91 percent) also indicated that their college used data on student 
outcomes (e.g., persistence, learning, degree attainment), not just enrollments, to evaluate 
academic programs and departments. A similar percentage (92.5 percent) indicated that each 
department or division in their college was required to set measurable goals and objectives as 
part of the planning process. Three fourths of administrators said that budget requests at their 
college must be supported by evidence that students would benefit as a result. 

In the Jenkins and Kerrigan broader analysis of the survey data, we found a surprisingly 
weak correlation between the extent to which administrators indicated that their college used 
data and research for program-related decisions and the frequency with which they themselves 
used data for decision making. This finding might be attributable to the fact that the 
administrator respondents included individuals working in areas not related to academics, who 
are probably less likely to use data as part of their jobs. Still, the survey also showed only a 
weak correlation between indicators of data use by individual faculty members and the extent to 
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which faculty indicated that their college overall used data on student outcomes to evaluate 
programs. Even weaker was the correlation between faculty data use and faculty members’ 
perceptions about the level of commitment by the college’s leadership to making decisions 
based on data and the clarity of the leadership’s vision on how to increase student academic 
success. These findings and the earlier ones about the variation in departmental practices 
suggest that the practices of individual academic departments have a greater bearing on the use 
of data by faculty members than do those of the college overall. 

As argued in the Jenkins and Kerrigan report on the overall analysis of the findings 
from the survey, the apparent disconnect between the extent of data use by faculty and 
administrators and the views and management practices of the college’s leadership calls into 
question a central premise of Achieving the Dream: that commitment by a college’s leadership 
and the way that a college approaches program evaluation, strategic planning, and budgeting are 
key to encouraging the use of data for improvement by college personnel. Survey findings 
suggest that leadership commitment and a data-oriented approach to institutional management 
may not be sufficient to encourage faculty and administrators to become more data-oriented in 
practice. Additional efforts at the department level are probably needed to change the behavior 
of faculty in particular. Indeed, we found that faculty in developmental education departments 
and for-credit occupational programs were more frequent users of data than were faculty in 
other types of departments, particularly those in general education. The greater intensity of data 
use in developmental education departments is perhaps not surprising given that improving 
developmental instmction has been a major focus of Achieving the Dream. The baseline 
evaluation of the first-round Achieving the Dream colleges found that the vast majority of 
participating colleges, if not all of them, were implementing some sort of strategy aimed at 
improving developmental outcomes. It may well be that a similar intensive focus on 
improving outcomes is needed to change practices and to influence the culture in other types of 
departments. 



Summary 

Most faculty members indicated that using data and research on students was part of 
their responsibility as faculty and that they had the skills needed to analyze data. A surprisingly 
high proportion of faculty in the PA, WA, and Round 1 colleges used data on student outcomes 
on a regular basis, but nearly one in three never reviewed data on student achievement gaps 
among different student groups. 

A majority of faculty at the PA, WA, and Round 1 colleges found data on student 
progression and achievement gaps at least somewhat useful in their jobs. Not surprisingly. 



'“Brock et al. (2007). 
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faculty who participated in Achieving the Dream activities at their colleges were more likely to 
find information on student outcomes useful and to use it in their jobs. 

The frequency with which faculty in the PA, WA, and Round 1 colleges used data for 
decision making varied by department, with those in general education on average less likely to 
use data on student outcomes in their work, while faculty in developmental and for-credit 
occupational programs were more frequent users of data and research. 

We found a much stronger relationship between data use by individual faculty and the 
extent to which their department used data on students for decision making than between 
faculty data use and the extent to which the college overall used data on student outcomes to 
evaluate programs and make decisions at the leadership level. Hence, commitment by top 
college leaders to data-based decision making and a data-oriented approach to institutional 
management may not be sufficient to encourage faculty to become more data-oriented in 
practice. Additional efforts at the department level are probably needed to change faculty 
behavior. 

Faculty at the Round 1 colleges were significantly more likely than those in the PA and 
WA colleges to indicate that they use data on retention and graduation rates frequently. This is 
consistent with the hypothesis that colleges that have been involved in Achieving the Dream 
longer should be more advanced in their use of data for improving student success. However, 
this finding cannot be seen as definitive evidence of a causal relationship between Achieving 
the Dream and more extensive use of data for improvement. CCRC and MDRC will have better 
evidence with which to examine the effect of Achieving the Dream on data use when we 
conduct a follow-up survey of faculty and administrators in the PA and WA colleges in two 
years, near the end of their participation in the initiative. 
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Chapter 4 



College Progress on the Initial Steps 
in the Process of Institutional Improvement 

Introduction 

During the planning year, colleges in Achieving the Dream are expected to begin 
carrying out the first three steps of the initiative’s five-step institutional improvement process. 
These steps are designed to engage college personnel in identifying areas where students are 
experiencing barriers to success and designing strategies to break down those barriers. These 
first three steps are: 

(1) Commit to improving student outcomes. 

(2) Use data to identify and prioritize problems. 

(3) Engage stakeholders in developing strategies for addressing priority 
problems. 

This chapter examines the progress that the PA and WA Achieving the Dream colleges 
made in implementing these three steps during their planning year (2006-07) and in the first 
year of implementation (2007-08). The chapter begins by examining the commitment of college 
leadership to the initiative and includes a description of how presidents and senior 
administrations organized and managed the initiative. It then explores how closely the colleges 
followed the initiative’s recommended process for identifying student achievement gaps and 
other problem areas. The chapter concludes by discussing the involvement of both internal and 
external college stakeholders in developing strategies to improve student outcomes. The 
progress of the colleges on step 4 (implement, evaluate, and improve strategies) and step 5 
(institutionalize effective policies and practices) are discussed in Chapters 5 and 6, respectively. 



Step 1 : Commit to Improving Student Outcomes 

The first step in the Achieving the Dream process for institutional improvement is for 
the college’s leadership to make a clear commitment to improve student outcomes, not just to 
increase enrollments. The Achieving the Dream framing paper, which provides the conceptual 
framework for the initiative, describes the role of college leaders as follows: 
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Institutional change succeeds when leaders frame inspirational values, 
engage others to bring the college’ s actions into alignment with those values, 
and institutionalize policies and practices that bring about positive results/' 

College leaders are expected to make the improvement of student outcomes a college 
priority and to communicate that priority to both internal and external college stakeholders. 
Leadership support for the initiative sends a signal to faculty and staff that Achieving the Dream 
is more than just another grant-funded project, and thereby encourages broad-based 
understanding and participation. Leadership commitment to the Achieving the Dream change 
process also implies a willingness to support changes in college policies and procedures and to 
make the resource investments necessary for improving student success, even in the face of 
competing interests and potential resistance from college stakeholders. 

Senior Leadership Commitment to improving Outcomes 

Senior leadership across the 13 PA and WA colleges was committed to making the 
improvement of student success — particularly for low-income students and students of color 
— a college priority. College presidents and senior administrators described a variety of 
ongoing efforts on their campuses to improve student outcomes, many of which were being 
funded through grants from federal programs such as TRIO, Title III, and Title V. College 
leadership at 10 of the 13 colleges indicated that they would likely use Achieving the Dream as 
a framework for current and future student success efforts. For example, the president of a WA 
college that made the initiative a college priority said: 

The types of student interventions that we are trying [to use] to affect student 
success are important to us. Achieving the Dream was helpful in pulling 
together all our interventions and activities to improve retention, throughput, 
and ultimately student success. 

Across all 13 PA and WA colleges, college leadership demonstrated a willingness to 
reallocate resources to improve student outcomes, including the hiring of additional institutional 
researchers. At the outset of the initiative, all but one of the presidents supported expansion of 
the use of data as a means to improve student outcomes and reduce achievement gaps. This 
president, with the encouragement of senior administrators and the college’s Achieving the 
Dream coach and data facilitator, became a supporter of data-driven decision making by the end 
of the first implementation year (2007-08). 

Eleven of the 13 college presidents were actively engaged in Achieving the Dream 
activities and were visible advocates for the initiative on their campuses, including regular 



"MDC (2006), p. 3. 
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participation in core team planning. Most presidents tapped senior administrators to lead the 
implementation of initiative, and they all kept their board of directors regularly updated on 
initiative activities throughout both the planning year and the first implementation year. At 
about half of the colleges, the research team found that direct presidential involvement was key 
to overall progress of the initiative. In other cases, colleges were able to make progress under 
the direction of senior administrators to whom the president had delegated responsibility for the 
Achieving the Dream work. 

Impact of Leadership Turnover on Commitment to Achieving the Dream 

The departure of a president or of senior administrators can threaten the leadership of 
college initiatives, as new leaders often shift college priorities according to their own agendas. 
One college applied to become an Achieving the Dream college during the president’s first year 
at the institution (2005-06), and another two colleges had presidents who left after joining the 
initiative. In all three cases, the new presidents embraced the initiative and indicated continued 
leadership commitment to the initiative. One of them expressed his enthusiasm for the 
initiative’s emphasis on building a culture of evidence to help disadvantaged students be 
successful: 



I think it is one mechanism for really focusing attention on individuals who 
are disadvantaged and need the kinds of opportunities that a community 
college provides.... It has awakened our faculty to understand that our 
population needs more opportunities.... Achieving the Dream is giving us a 
new way to look at ways to meet the needs of students and opening our eyes 
to a culture of evidence and the importance of assessment. We have to do 
more and more of that. It has the potential for far-reaching importance. 

The research team found that each of the three new presidents was a more forceful 
advocate of using data for institutional improvement than his or her predecessor. One, who 
suggested that the initiative would provide data that would inform the college’s strategic 
planning, was committed to improving student outcomes and was particularly focused on 
improving student success rates in developmental education. He embraced the initiative’s focus 
on student outcomes data, saying: 

We had no data about students. Achieving the Dream for a new college 
president faced with challenges was a gift from God, a bully pulpit — [and I] 
didn’t even need to bully. Faculty and staff were eager to make changes, but 
never had been given permission. Achieving the Dream gave us a structure: 
to be methodical, to substantiate what we knew. It changed the culture from 
making assumptions to making decisions. 



53 




While presidential commitment to the initiative did not falter in the midst of leadership 
turnover at the colleges, the research team did find that a change in leadership delayed 
implementation of the initiative at a couple of the colleges. For example, administrators and 
faculty at one college were reluctant to commit to working on the initiative because frequent 
presidential turnover there had made them apprehensive about changing leadership priorities. 
An administrator at one college said that as a result of the leadership turnover: “Everything is up 
in the air. Nothing is definitive. Hopefully something can take place. Everything has been 
uncertain.” 

Incentives for Leadership Commitment 

Achieving the Dream, in seeking to change the culture of community colleges and 
make long-lasting improvements in institutional practices, expects college personnel to invest 
significant time and effort. College leadership clearly welcomed the funds attached to the 
initiative, yet presidential commitment to the initiative was not driven by the money. None of 
the 13 college presidents suggested that the initiative’s grant money was an incentive for the 
college to participate. Indeed, almost half of the colleges had already invested a substantial 
amount of their own funds in the initiative by the time of our visit. As mentioned in Chapter 1, 
one PA college used its own resources to fund its participation. The factors that did encourage 
presidents and senior administrators to support and promote the initiative at their colleges are 
described below. 

Consistency with college goals 

Presidents at each of the 13 colleges viewed Achieving the Dream as consistent with 
institutional goals and current efforts to improve student success. The boards of trustees at all of 
the colleges had made a commitment to improve student outcomes, and the initiative was 
viewed as consistent with board priorities. The view of one college president in PA was typical. 
This president had made student success a centerpiece of her administration prior to the 
college’s involvement with Achieving the Dream. Under her leadership, the college was 
looking at a broad range of issues related to student success, including developmental 
education, instructor grading policies, first-year experience, and student services. The president 
viewed Achieving the Dream as a natural fit with the student success agenda she was 
spearheading at the college. 

Involvement with a high-profile national Initiative 

Participation in a high-profile, national initiative was another incentive for leadership 
support for Achieving the Dream. Several college leaders discussed how joining Round 3 of the 
initiative gave their institutions additional status, and they were proud that their schools were 
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selected to participate. Other college presidents focused on the support the initiative provided to 
colleges to improve student outcomes. The president of one PA college said: “For us, Achieving 
the Dream is not about the money. It’s about having the backing of a prestigious outside 
initiative.” This president said that joining a national student success initiative had enabled the 
college to have conversations with faculty and staff about student outcomes without creating the 
perception that the administration was blaming the faculty for poor student outcomes. The 
president said, “Achieving the Dream is a way to have a conversation by having someone out of 
house asking the questions, asking us to look at the data. That’s better than trying to have that 
conversation in-house.” A president in WA expressed a similar sentiment: 

The other piece that was helpful to our college and community was to have 
an external entity say that it was important to serve the neediest and that there 
were parts of the community that weren’t being served. Having an external 
group saying that consistently helps us focus. 

Several college presidents used the Achieving the Dream coaches and data facilitators 
to engage faculty and staff in conversations about achievement gaps among subgroups of their 
students. The coaches and data facilitators were able to present poor student outcomes as an 
issue confronting colleges across the country, rather than as the fault of one particular 
institution. Several college leaders and senior administrators suggested that conversations about 
poor student outcomes would have been more difficult without the support of the initiative. 

Provision of a roadmap to achieve college goals 

The majority of college presidents and senior administrators indicated that Achieving 
the Dream provided a helpful roadmap to improve student performance and close the 
achievement gap at their institutions. For example, the president of a PA college was using 
Achieving the Dream to increase faculty engagement in the college’s long-term strategic 
planning process. Senior administrators at this college had long been committed to using data to 
inform strategic planning efforts, yet faculty and staff had generally been excluded from these 
efforts. Achieving the Dream was seen by the president as an opportunity to engage a wider 
group of stakeholders in the planning process and expand the use of data across the college. 

Achieving the Dream was viewed by many college leaders as helping their colleges 
prepare for and respond to a changing student body. College personnel in both states described a 
noticeable shift in the student demographics and in the readiness of their students for college- 
level work. A PA college president said: 

We are seeing a greater influx of minority students. We found fewer minority 
students are [graduating]. We’re wrestling with how to attack the issue and 
issues of retention. Achieving the Dream fits with the issues that are arising. 
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In WA, a college with an increasing percentage of low-income and English as a Second 
Language (ESL) students was struggling to move students from ESL and adult basic education 
(ABE) into college-credit courses. The college was clearly dedicated to better understanding 
and serving these students, and the president and senior administrator viewed Achieving the 
Dream as a guide to improvement. The interim vice president of instmction described the 
importance of transitioning ABE and ESL students into career and technical education (CTE) 
programs at the college: 

The demographics of the area have continued to change. Our assumption is 
that ABE/ESL will grow by a least 10 percent every year into the foreseeable 
future. Transfer enrollments and CTE enrollments are flat.... If the technical 
faculty want to think about their market, they need to think about this 
population. There is a realization that we are different than we used to be, 
and people are thinking about what this means for the programs on campus. 

These students [ABE/ESL] become a pool of potential students for their 
programs. It is beginning to be viewed as central to the college. 

Synergy with accreditation standards 

Presidents and senior administrators interviewed at 10 of the 13 colleges thought that 
Achieving the Dream would help prepare their institutions to comply with regional 
accreditation standards. In both states, the initiative’s emphasis on using data to revise college 
practices and policies to improve student outcomes was reinforced by accreditation standards. 
The PA and WA colleges were in various stages of the reaccreditation process; some colleges 
had recently completed their reaccreditation efforts, while others were preparing for an 
upcoming accreditation visit. 

Colleges that had already had a recent accreditation team visit indicated that the 
experience had helped prepare their institutions for data-driven decision making. A WA college 
president said that all of the WA colleges were “getting scalded by accreditation,” and viewed 
Achieving the Dream as helping colleges comply with accreditation standards. Another WA 
college that had recently gone through the reaccreditation process received a recommendation 
regarding insufficient academic advising. As part of its Achieving the Dream efforts, the college 
used data on its students to improve the advising process. The college redesigned a student 
success course, and preliminary evaluation data suggested the new course had already led to 
improved student retention. Senior administrators said the accreditation finding provided 
additional impetus to focus initiative efforts in that area. Similarly, a PA college received a 
recommendation from its accrediting agency in 2005 for inadequate student outcomes 
assessment. Prior to joining Achieving the Dream, its institutional research (IR) department had 
begun working with faculty across the academic departments to strengthen learning assessment. 
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The college’s participation in Achieving the Dream was viewed by both administrators and 
faculty as furthering those efforts. 

Administrators at several colleges with an upcoming reaccreditation visit said 
Achieving the Dream, with its emphasis on building a culture of evidence, would help them 
prepare for the visit. College leadership suggested that the regional accrediting bodies for both 
PA (Middle States Association of Colleges and Schools) and WA (The Northwest Commission 
on Colleges and Universities) are increasingly demanding evidence of measurable student 
outcomes. In PA, one college already decided that Achieving the Dream would provide the 
framework for the college’s reaccreditation efforts. In WA, several college personnel suggested 
that the Achieving Dream model of institutional improvement involved a more rigorous 
analysis of student outcomes data than was required by accreditation; an administrator with 
many years of experience with accreditation said that the Northwest Commission had weaker 
institutional effectiveness standards than other regional accrediting bodies. However, according 
to two WA college presidents, the Northwest Commission is in the process of revising its 
accreditation standards, and the new standards are expected to have a greater focus on the use of 
data to measure and improve student outcomes. The president of one WA college said 
Achieving the Dream would help the college in future accreditation visits: 

Everything we’re doing with Achieving the Dream is a great asset for 
accreditation. The Northwest accreditation is going under major 
transformation — details of how it ends up are unclear — but Achieving the 
Dream will have a real positive impact. 

Alignment with state higher education goals 

State policy also positively influenced leadership commitment and support for 
Achieving the Dream. This was particularly the case in WA, where new policy initiatives from 
the State Board for Community and Technical Colleges (SBCTC) are spurring colleges to focus 
on improving student outcomes using performance data. Most notably, the Student 
Achievement Initiative provides financial rewards to colleges for increasing the rate at which 
their students attained key “achievement points” in four categories: (1) making gains in adult 
basic skills and pre-college remedial courses; (2) completing a college-level math course; (3) 
earning college credits; and (4) completing a certificate, degree, or apprenticeship training. Both 
Achieving the Dream and the Student Achievement Initiative encouraged colleges to look at 
barriers to student progress and then to develop strategies to overcome them. The president of 
one WA college argued that Achieving the Dream, with its focus on the creation of a campus- 
wide culture of evidence, would help students progress in their educational programs — a 
primary goal of the Student Achievement Initiative. At another WA college, the director of 
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institutional research described how Achieving the Dream and the Student Achievement 
Initiative were both pushing the college toward more systematic data analysis: 

Achieving the Dream’s timing is perfect because we have in WA the Student 
Achievement Initiative ... which is fascinating and has all sorts of IR 
components ... thinking of IR as a strategic resource and something you 
would do deeply in a very purposeful and analytical way, rather than just as a 
reporting and record-keeping function. 

Furthermore, senior administrators at the college began to focus their Achieving the Dream 
efforts on improving student performance in developmental math, one of the “achievement 
points” in the Student Achievement Initiative. 

While most WA college presidents commented on the synergy between Achieving the 
Dream and the Student Achievement Initiative, it is important to note that not all presidents 
were supportive of the new state initiative. One president had a favorable view of Achieving the 
Dream, saying the initiative allowed colleges to focus on their own particular student needs and 
problem areas. However, the president argued that Achieving the Dream strategies, designed to 
address students needs, could potentially harm a college’s ability to accme achievement points 
under the Student Achievement Initiative. For example, the president highlighted the points 
received for students who complete a certificate or degree, and suggested that it sometimes 
makes more sense for students to transfer prior to receiving an associate degree: “Good advising 
is not always telling students to complete the degree.” 

In PA, the research team saw less evidence of any influence or overlap between state 
policy and Achieving the Dream. In contrast with WA, PA did not have a state-level governing 
or coordinating board to provide oversight and direction to the state’s community colleges. One 
dean of students in PA did suggest that his college was able to use data required for state 
reporting in its Achieving the Dream efforts. The college, which had been required by the state 
for more than 30 years to collect student attendance records, began using the state-mandated 
attendance database as part of an early-alert strategy to identify and reach out to students who 
were struggling with poor attendance and course performance. According to the dean, the early- 
alert strategy was an example of the college’s creativity in learning how to use state-required 
compliance data to improve student success. 

Communication about Achieving the Dream 
to internai Coiiege Stakehoiders 

Achieving the Dream expects college leaders to communicate the Achieving the Dream 
vision widely within the college. The PA and WA presidents and senior administrators used a 
variety of communication channels to inform the college community about initiative goals and 
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values, including college-wide forums such as fall convocations, faculty in-services and other 
professional development days, email alerts, data briefs, and featured presentations by 
Achieving the Dream coaches and data facilitators. 

These communication efforts were fairly successful in raising awareness and 
understanding of the initiative among college personnel. In over half of the colleges in both PA 
and WA, faculty and staff interviewed by the research team suggested that a substantial number 
of their colleagues understood both the goals and details of the initiative. 

Organization and Management of the initiative 

Achieving the Dream expects college leaders to organize teams of college personnel to 
oversee the initiative’s process of institutional improvement. During the planning year, colleges 
were to form separate core and data teams to guide their work on the initiative. The core team’s 
function was to lead the policy and institutional change work, while the data team was expected 
to collect and analyze student outcomes data to support the work of the core team. 

Core and data team structure 

All of the colleges began their Achieving the Dream work with a core team, which 
generally involved a broad cross-section of college personnel, including faculty leaders, mid- 
level administrators, and counselors. Almost all of the college presidents put senior 
administrators in charge of leading or co-leading core team activities. 

All but two colleges also began the planning year with separate data teams, and, with 
one exception, they included non-IR personnel. One of them, which did not have a separate data 
team, chose to start its planning year with a combined core and data team called the Achieving 
the Dream steering committee. The steering committee reviewed data, discussed possible pilot 
programs, and proposed implementation strategies. At the end of planning year, the steering 
committee created a separate data taskforce which continued to meet regularly. The other 
college chose to establish not just one data team, but a team for each of the five main Achieving 
the Dream performance indicators. Furthermore, during the implementation year, each of the 
college’s pilot strategies had a designated data person who worked with the office of 
institutional effectiveness. 

While the core teams were still functioning during the implementation year, 7 of the 13 
colleges did not have functioning data teams at the time of the research team visit. At 2 of these 
colleges, the data teams were on hiatus until results from strategy pilots were ready to be 
analyzed, or until vacancies in initiative leadership were filled. Another 2 colleges in WA 
disbanded their data teams and incorporated the data team responsibilities into the work of their 
IR offices. Because of significant leadership transitions during the course of the planning year. 



59 




one PA college made the decision to merge the core and data teams. Many faculty who were 
originally involved drifted away and the remaining team was not terribly active while they 
waited for a new president to be appointed. 

Two colleges created a new permanent structure that absorbed the responsibilities of the 
data team. One of the colleges created an office of institutional effectiveness (IE). At the other, 
the data team, which included faculty, administrators, and IR staff, became a permanent 
stmcture known as the Institutional Assessment Council. The president described how 
integrating both the core and data teams into the college’s permanent operations served as a 
signal to college personnel that the initiative was not a temporary, grant-funded project: 

We’re looking at Achieving the Dream as a vehicle to accomplish what 
we’re already trying to do. We’re using our planning council as the core team 
— to negate the perception that this is an add-on. They’re looking at the data, 
persistence, what’s going on in programs. There’s a logical fit.... We’re going 
to create an assessment council that’s part of the governance stmcture of the 
college. The data team was moved into this. These will just become 
embedded in the culture. We’re already starting to get there. 

One WA college was considering making its data team a permanent committee. 
According to the director of institutional research, the president viewed the data team as critical 
to facilitating data-driven decision making across the institution: “The data taskforce is the first 
step in [institutionalizing a culture of evidence]. The president has asked for the data taskforce 
to become permanent.” 

Core and data team leadership 

The research team found that stable leadership of the Achieving the Dream teams by 
respected senior administrators helped move the initiative forward. By placing senior 
administrators in charge of the core team, the presidents signaled to faculty and staff that the 
initiative was a college priority. For example, one president gave a senior administrator at the 
college the task of co-chairing the core team and of helping lead the initiative. This senior 
administrator was well respected at the college and seen as effective in building consensus for 
change and then implementing good ideas. Both the senior administrator and the other co-chair 
of the core team (the dean of student affairs) established an open and transparent process that 
encouraged faculty, staff, and students to become involved. 

At a few colleges there was turnover among senior administrators who led the core 
team. At one of them, despite the strong commitment of the president to Achieving the Dream, 
progress on the initiative essentially came to a standstill because of turnover among senior 
administrators responsible for its day-to-day leadership. A staff member explained, “There was 
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a major stumble with all the staff changes. The person who was in charge [of directing the 
initiative] retired. ... There was a loss of momentum and people got dispirited.” 

Engagement of faculty in core team activities 

Engaging faculty and faculty union leaders in core team activities helped build college- 
wide support for the initiative. The president of the faculty union at one WA college was a co- 
leader of the core team. The involvement of a faculty leader with wide informal networks of 
support throughout the college helped shape campus- wide opinion of the initiative. The college 
also rotated the membership of the core team to facilitate understanding of the initiative and 
participation among a broad segment of the college. Both of these tactics were successful in 
raising awareness and faculty buy-in to the initiative. 



Step 2: Use Data to Identify and Prioritize Problems 

Achieving the Dream’s second step in building a culture of evidence calls for colleges 
to diagnose problems in student achievement and identify priority areas for student success 
interventions. The colleges were expected to use longitudinal student cohort data and other 
evidence to identify gaps in achievement among different student groups as well as “leakage 
points” where students stmggled or dropped out. A key assumption of this approach is that once 
faculty and staff see that certain groups of students are not doing as well as others, they will be 
motivated to address barriers to student success. 

Process for Identifying Student Achievement Gaps 

Colleges in both PA and WA closely followed the Achieving the Dream process of 
identifying student achievement gaps and other problem areas. All 13 colleges relied on an 
analysis of their own college’s data as the primary means of identifying gaps in student 
achievement. Twelve of them used longitudinal cohort analysis to identify problems, and all the 
colleges disaggregated their data analyses by student race and ethnicity to identify achievement 
gaps. Most of the PA colleges and all of the WA colleges also used the Achieving the Dream 
database to help with problem identification. 

The colleges collected qualitative data to identify problem areas through both student 
and faculty focus groups and student surveys. All but one college used student focus groups or 
student surveys to identify problems in student achievement. The Community College Survey 



*^JBL Associates, a higher education consulting firm, is compiling the Achieving the Dream database with 
data provided periodically from all the participating colleges for the purpose of measuring progress. 
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of Student Engagement (CCSSE)^^ was a particularly popular tool used by the colleges to 
measure student engagement. All of the PA colleges and three of the WA colleges used CCSSE 
results to identify problems in student achievement. One WA college was waiting for its base- 
year CCSSE results at the time of the research team visit. So, while the college had not yet used 
the results to identify problem areas, it was planning to incorporate CCSSE into its future 
planning efforts. 

The Achieving the Dream process of identifying problem areas encouraged many 
colleges to begin to shift their approach toward measuring student outcomes. Eike many 
community and technical colleges across the country, the majority of the WA and PA 
Achieving the Dream colleges had little experience in using data on students to drive decision 
making. Three colleges did not have an institutional research department prior to their 
involvement with Achieving the Dream; institutional research at these colleges primarily 
consisted of compliance reporting for state and federal agencies and for accreditation purposes. 

Even among the colleges that did have IR offices, the majority did not use data to 
systematically evaluate student success efforts prior to joining Achieving the Dream. Eor a PA 
college, joining the initiative led to the adoption of a systematic approach to monitoring student 
outcomes for the first time. The college had experienced significant student enrollment growth 
in recent years and measuring student outcomes was at best a secondary focus at the institution. 
As part of the process of identifying problem areas, the college embarked upon extensive 
student cohort tracking, disaggregating student outcomes by race and ethnicity. The college 
focused on tracking student progression from developmental education to college-level courses. 
It analyzed CCSSE data and conducted focus groups with both student and faculty to better 
understand the challenges that students were facing in their developmental education courses, 
and with developmental math in particular. This data analysis led to important findings and 
recommendations for change. 

Colleges more experienced in sophisticated data analysis also benefited from the 
initiative’s emphasis on disaggregated student outcomes data. Eor example, one college had 
long used data to make decisions, but in a limited fashion. The college had not previously 
disaggregated and analyzed its data until joining Achieving the Dream. The process became 
much more systematic after joining, and there was an expectation that decision makers at the 
college would use data to inform decisions that impact student success. According to its IR 
director: 



*^The Community College Survey of Student Engagement (CCSSE) is a national survey administered to 
community college students that assesses institutional practices and student behaviors that are correlated with 
student learning and retention. CCSSE was established in 2001 as a project of the Community College 
Leadership Program at the University of Texas at Austin. 
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There was no systematic plan for identifying achievement gaps. As problems 
or specific issues emerged, like students on probation, research would head 
in that direction, but it wasn’t a systematic review process. The 
implementation of our data team really got our analytical resources together 
to look at our data, disaggregate it, and to look at our achievement gaps and 
to decide on what interventions we wanted to adopt. The Achieving the 
Dream structure focused us on the gaps we want to address. 

Institutional Research Capacity 

College leadership signaled its commitment to more systematic data analysis by hiring 
additional personnel to improve IR capacity. Just over half of the colleges hired new personnel 
to staff the institutional research offices. Two of the three colleges that did not have an IR 
department prior to joining the initiative established and staffed institutional research (or 
institutional effectiveness) offices. An additional four colleges — one in PA and three in WA 
— added institutional researchers to their existing IR or IE department. 

Despite the success that the initiative seemed to have had in moving colleges toward 
more systematic data analysis, IR capacity remained a clear obstacle for many colleges as they 
tried to accurately identify and diagnose problem areas. The IR staff members at the two 
colleges that established an IR function because of Achieving the Dream had a steep learning 
curve as they sought to develop their data analysis capabilities. For example, one of the colleges 
had come a long way in developing institutional research capacity, and faculty and staff 
welcomed the new IR office with numerous data requests. A research analyst at the college said 
that the “need to measure and have [data] input infiltrates everything.” Yet, the new director 
responsible for data analysis came out of a grant writing and compliance background and lacked 
formal training in data and statistical analysis. The college will likely struggle as it continues to 
develop and implement a coherent and focused research plan. The research analyst described 
the challenge by saying: “I feel overwhelmed with the work demands, but we’re moving at a 
fast pace and it’s exciting. We’re building the car and driving it at the same time.” Even 
colleges with established IR or IE functions were challenged. The director of a three-person 
institutional research office described in blunt terms multiple challenges: 

I would say that right now our IR function has a horrible internal reputation 
as far as people even asking them for anything. Part of it is the labor it takes 
to get the information out. Part of it is the lack of enough people working in 
that area and a lot of it is also the organization issues and the prioritization 
issues and communication issues we have that are significant. 
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In WA, the State Board for Community and Technical Colleges (SBCTC) has a 
national reputation for having a sophisticated state-level data system coupled with outstanding 
policy research capacity. In fact, it used that data capacity, which can track individual student 
progress through the system, to design and implement the Student Achievement Initiative. Yet, 
the WA Achieving the Dream colleges as a group did not have stronger IR or IT capacity than 
the PA colleges. Despite the Washington’s reputation for sophisticated data systems, the 
individual colleges tended to find the legacy data system shared by the community and 
technical colleges in the state very difficult to access and use. It took a highly experienced data 
manager to be able to manipulate the system in order to conduct analyses beyond those 
provided by the state. 

Impact of Personnel Turnover 

Personnel turnover among staff responsible for institutional research delayed the data 
collection and work of the data teams at four of the colleges. At one institution, the college’s 
institutional researcher left after doing some initial analyses tracking the progress of students 
over time. According to one faculty member, “People were really excited that we were going to 
do something for that cohort. Then we lost our institutional researcher, her assistant, and then 
the VP. Some of the committee members turned over.” Longitudinal cohort tracking was 
aborted shortly thereafter during the planning year. 

At three of these four colleges, however, the resignation of IR personnel proved to be 
only a slight setback; the colleges’ IR departments emerged stronger than before with new staff. 
The fourth had difficulty hiring IR staff due to its location in an economically depressed area, 
and was still trying to h ir e at the time of the research team’s visit in spring 2008. 

Presentation of Data Analysis to Faculty and Staff 

All 13 colleges presented the results of their analysis of achievement gaps to faculty and 
staff across their institutions, though the sharing of student outcomes data with faculty was not 
without risks. Achieving the Dream’s model of institutional change, which calls for the use of 
data to identify and prioritize problem areas, can threaten longstanding ways of conducting 
business, and brings college personnel face-to-face with potentially unflattering data on student 
performance. A WA president explained the challenge of presenting faculty and staff with 
evidence of low success rates among their students: “If you have worked for years and feel 
confident and then you are told you aren’t being successful, it’s not easy to take.” Most colleges 
did not have much prior experience sharing student outcomes data with faculty and student 
services staff, particularly data on students across the college rather than in specific programs. A 
faculty member at one college said that, in the past, data analysis was done primarily by 
individual departments: “We were a bunch of little chimneys. We only looked at our own data 
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when we did our program reviews. I like being able to look across and see what others are 
doing.” 



Colleges used various means to communicate the results to faculty and staff, including 
during president’s day and other college- wide events, through research briefs, and in campus 
publications. At about half of the WA colleges and almost all of the PA colleges the realities of 
poor student outcomes came as a surprise. The president of a PA college said: “When we started 
reporting how many students we were losing and graduation rates, faculty were in denial that it 
was happening in their own programs.” The longitudinal data analysis at one of the WA 
colleges revealed that, while the student pass rate was high in the first college-level English 
course, students were not progressing to the next English course. According to one faculty 
member: 



English was stunned by the falloff between English 101 and 102. People 
were hitting the wall. We’re a very diverse school. I was stunned at the high 
attrition rate of our African-American students, which is a large part of our 
student body. ... We had a sense that some of these students were weak going 
into the next course. But it blew us away that so many would not be 
completing the follow-up course. 

The presentation of low student outcomes data was generally met with genuine interest 
and reflection by faculty and staff. At only a few institutions did some college personnel try to 
explain away the poor outcomes by saying that the students were responsible for their own lack 
of success or suggesting that other departments or divisions at their colleges were at fault and 
needed to improve. Instead, at most of the colleges, the faculty and staff interviewed by the 
research team seemed to believe that improving the success of their students was within their 
control. Eurthermore, at every college, faculty and staff indicated that the identified 
achievement gaps and problems areas in student outcomes provided motivation to improve and 
prioritize student success strategies. Eor example, at one college where longitudinal data 
analysis was done for the first time during the Achieving the Dream planning year, significant 
student achievement gaps came as a surprise. The data prodded the college to discuss the 
barriers to success among African-American students and other student subgroups, an issue that 
is often ignored. 



Step 3: Engage Stakeholders in Developing Strategies 
for Addressing Priority Problems 

The third step in the Achieving the Dream institutional improvement process is 
engaging internal and external stakeholders in the development of new student success 
strategies. Achieving the Dream encouraged the colleges to involve as many voices as possible 
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in the process, including those of faculty, student services staff, community representatives, and 
students. This is a tall order for colleges that are already stretched thin serving disadvantaged 
students. Yet the buy-in of faculty and staff on the front lines of working with students is critical 
for effective and long-lasting student success interventions. A WA president explained the 
importance: 



Anytime you ask people to do things differently and develop a different 
attitude toward students, if you want to be successful, you have to have 
people involved on front lines working with students really buy into it. Any 
top-down effort is doomed to failure, so you have to find ways to work with 
people using the data-driven approach to work collectively to come up with 
these strategies. 

Receptiveness to the Initiative by Faculty 

Faculty generally had a favorable view of Achieving the Dream goals and principles. 
The initiative, by tracking student progression within and across courses and programs, was 
seen by many instmctors and counselors as a particularly effective means of helping them 
identify areas to improve. Faculty and staff at some of the colleges suggested that Achieving the 
Dream, as a high-profile national initiative, spurred their presidents and senior administrators to 
support their efforts to get more information on their students. For example, a math faculty 
member at a WA college said that his department had been working for several years to use 
course-level data on student outcomes to improve the math curriculum and refine teaching 
methods. According to this instructor, “[Achieving the Dream has] given us legitimacy with the 
president because if we said it’s an Achieving the Dream initiative, it made it a priority.” 

Some college personnel interviewed by the research team were concerned that 
Achieving the Dream would be just another administrative program du jour. A vice president of 
a PA college described the reaction to Achieving the Dream from many among his college’s 
faculty and staff as: “Here comes another program.” He further said that they were reluctant to 
commit to the initiative “because no one ever looked at what we did with our existing 
programs.” 

The president of one PA college suggested that younger faculty members would be 
more willing to embrace new learning and teaching strategies than older, more entrenched 
instmctors: 



When you ask faculty to alter their classroom practices, the way they work 
with their students, that’s a challenge. Fortunately, 50 percent of the faculty 
has been hired within the last five years, so they are not as firmly entrenched. 
They seem open to look at things differently. 
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In some cases, receptiveness to the initiative was affected by the relationship between 
the administration and faculty. At one institution with an institutional culture that promoted 
healthy collaboration between administrators and faculty and student services staff, the initiative 
was well received. The core team leader, a senior administrator, successfully framed the 
initiative as integral to daily activities of faculty and staff. He described his approach to 
informing the college’s faculty about the initiative: 

One selling piece I did in the beginning is that Achieving the Dream is not 
one more thing on your plate. It is really a way to frame things we are 
already doing, and create evidence-based decision making. There was a lot of 
buy-in. I must have said it ten times in different meetings. The faculty here 
have bought into the idea of a larger faculty responsibility. 

At a few schools, conflict between the administration and faculty negatively affected 
the faculty’s receptiveness to Achieving the Dream. Several faculty involved with the initiative 
at one college perceived it as a top-down administrative project, even through there was fairly 
broad faculty representation on the initiative’s core team. The faculty and staff’s reaction was 
clearly part of general dissatisfaction with faculty-administration communication at the 
institution. At another college, there was a clear divide between the administration and faculty 
that hindered collaboration, with any initiative being driven from college leadership viewed 
with considerable suspicion by faculty. 

Faculty and Staff Concerns with Achieving the Dream 

At a handful of colleges faculty and/or staff expressed concerns about the potential 
negative consequences of their institution’s involvement in Achieving the Dream. 

Time commitment 

At almost half of the colleges, some faculty were concerned about the time 
requirements for the initiative. At colleges where faculty were stretched thin by the demands 
from students and administrators, the perception that Achieving the Dream was an additional 
work burden created resistance to participation. Veteran faculty members at one college 
reported seeing many student success programs come and go over the years, and they were 
reluctant to invest time in an initiative that may also be short lived. An instmctor at this college 
said: 
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Faculty are concerned about investing in initiatives. In the past when we have 
done so much with learning communities [and] active learning, then nothing 
changes. Faculty are afraid that we will invest all this energy that we don’t 
really have in Achieving the Dream. And then next year we won’t have the 
resources to follow through. 

A faculty colleague at the same college said: “I’m passionate about [Achieving the 
Dream]. But it is a challenge trying to fit this into an overload situation.” Some saw the 
initiative as an add-on to their current job responsibilities, clearly secondary to their mission of 
teaching and serving students. For example, in WA, a faculty member said: “I think it is a 
reality that should be acknowledged that it [Achieving the Dream] takes time and it isn’t our 
primary function.” 

Lowering of standards 

At several colleges, some faculty expressed concern that improving student success 
would mean lowering standards. For example, at one PA college there appeared to be a rear 
guard of faculty who took issue with the Achieving the Dream premise regarding the potential 
of colleges to improve student success. In their view, the lack of student success represented 
inadequate high school preparation, and Achieving the Dream’s focus on student persistence 
threatened to “dumb-down” the academic programs. Similarly, an instmctor at a college in WA 
voiced the concern heard among his colleagues that “if students don’t get it, the college 
shouldn’t dummy down our program to make completion statistics.” 

Preferential treatment for certain students 

Some faculty and staff at a few of the colleges expressed concern that the initiative 
encouraged preferential treatment for certain groups of students. A WA college faculty member 
said: “Faculty are uncomfortable with specific programs for certain groups of people. One 
strategy for all is preferable. We need to have uncomfortable conversations about that.” At a 
second WA college, part of the focus of the initiative was on improving the success of Latino 
students, and some faculty and staff raised concerns that it wasn’t inclusive of all of the 
college’s student body. An English faculty member who wasn’t involved in designing the 
college’s strategy focused on Latino students said: 

We have a highly resistant faculty. Some are curmudgeons, people who resist 
anything — a basic resistance to political correctness.... Political correctness 
is a conversation stopper.... Faculty see Achieving the Dream as looking at 
some obstacles for some students. We need to [address] obstacles that are 
affecting all students. 
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Process for Designing Strategies to Address Achievement Gaps 

Colleges largely followed the Achieving the Dream planning process in the design of 
new strategies. At a few colleges, personnel were anxious to implement student success 
interventions soon after their college joined Achieving the Dream. A developmental reading 
faculty member at one of them who also sat on the core team described how she had to remind 
her colleagues of the necessity of sound data analysis before crafting student success 
interventions: 

In the beginning faculty didn’t understand how this would progress — 

[They] wanted to get to implementation much more quickly. [I] needed to 
explain that this was data driven and we needed time to do the data analysis. 

Most colleges did not develop improvement strategies until after analyzing their data. 
Only three had clearly identified the strategies they wanted to implement prior to an analysis of 
the problem areas. Three PA colleges and five WA colleges relied on their own data on student 
outcomes as the primary means of formulating their initiative strategies. Colleges also tapped 
national research on effective student success interventions to inform their strategy 
developments. Colleges in both PA and WA benefited from the experiences of colleges that had 
joined the initiative in previous years. Teams from all 13 colleges participated in the Achieving 
the Dream Strategy Institute, which was also well attended by teams from previous rounds. 
Several colleges took note of mistakes and successes of these earlier round colleges, and many 
of the strategies adopted in WA and PA were informed by presentations at the Strategy Institute. 
In addition, college personnel at several institutions reported using the Achieving the Dream 
website as an additional resource to support strategy development. 

Involvement of Faculty and Staff in the Planning Process 

Full-time personnel 

Seven of the 13 colleges engaged faculty and staff on a wide scale in the process of 
using data to develop student success strategies. At one college with a tradition of being data 
driven, pronounced institutional silos had previously limited collaboration on strategies to 
improve student success. While the college clearly had more work to do in breaking down those 
silos, more faculty and staff became involved in examining student success data and seeing the 
differences in student performance by subgroup. The college created five cross-cutting work 
groups during the planning year, each focused on one of the initiative’s performance indicators. 
These teams analyzed data provided by the institutional research staff and then recommended 
strategies to an initiative planning team. Participation in these five work groups included over 
90 faculty and staff members from across the college, and the process encouraged widespread 
faculty involvement in issues regarding student success. The initiative at this college also 
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increased the visibility and stature of the IE team and the student affairs staff. As a whole, the 
college became more focused on student success and on thinking of new strategies to help all 
students be successful. The provost said: “I think there’s a creative energy that is different 
around possible solutions.” 

Another college made significant progress in engaging faculty in the planning of a 
developmental education strategy for the initiative. The college brought a group of faculty 
together to work on the cut scores of the college’s developmental education placement exam, 
the ACCUPLACER. The developmental education faculty reported that this was the first time 
that they had been asked to provide input about the placement exam. The faculty had not 
previously analyzed the placement scores of incoming students, nor the placement levels or 
student grades in developmental courses. The process helped both developmental and college- 
level faculty understand the purpose of the placement exam. The faculty group determined that 
the existing course structure was not effective and began to reorganize the math sequence, and 
they were planning an evaluation of student outcomes under the new structure. 

A third college had a combined core and data team, called the Achieving the Dream 
steering committee, during the planning year. During the implementation year, an additional 
developmental education taskforce was created with faculty and administrators from this 
steering committee. The new group had visibility across the college and appeared to have 
enthusiastic members who showed real interest in making developmental education more 
successful. This developmental education taskforce, which included representatives from both 
faculty and student support staff, created an avenue for continued, broad, and engaged 
discussion about issues relating to student success. 

Yet, at about half of the colleges in both PA and WA, interviewees indicated that a 
relatively small number of faculty and staff were actively involved in analyzing the data on 
student success and identifying strategies for improvement. Administrators, faculty, and staff at 
several colleges described the amount of time and effort required by the initiative, and 
suggested that limited broad-based participation. Only two colleges gave faculty release time 
from instmction to facilitate their participation in initiative planning. One of them provided 
release time to three faculty members to participate on the core team. The college realized early 
on that participation in the core team would require a substantial set of responsibilities and it 
wanted full participation from faculty representatives (one of whom also served as president of 
the faculty association). At the other college, faculty were fully engaged from the start of the 
initiative to design how the college would respond to the gaps in student achievement. As the 
instructional vice president put it: “If faculty need to be involved at the end, they need to be 
involved in the beginning to help shape the solutions.” 
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Adjunct instructors 



Most colleges struggled with how to engage adjunct instructors in initiative planning. 
Indeed, only two colleges actively sought to engage adjuncts in the planning process. As with 
other initiatives, scheduling and college expectations regarding adjunct participation on campus 
committees or at meetings were barriers to their involvement with the initiative. 

Board, Student, and Community Engagement 

College presidents kept their boards of tmstees regularly informed of initiative 
activities and a few colleges included board members on their core teams. However, most board 
members were not routinely engaged in the initiative. Ultimately, it is the responsibility of 
tmstees to hold college leadership teams accountable for improved student outcomes and to 
require that college programs and processes contribute to meeting public needs. Thus, if the 
goal of Achieving the Dream is to bring about sustainable institutional transformation, closing 
achievement gaps, and improving student success for at-risk populations, trustees need to be 
better engaged at the initiative policy level. 

Similarly, while student focus groups contributed insights into problem areas at most 
colleges, no college chose to engage students directly in designing strategies. The lack of 
student involvement in designing strategies was not too surprising, as the vast majority of 
community and technical college students commute to campus, and they face significant 
challenges balancing school, work, and family responsibilities. 

Community members or groups were rarely informed about the initiative or engaged in 
its activities, other than occasional presentations by a college leader in the community. Only a 
handful of colleges reported to outside stakeholders about their Achieving the Dream findings. 
In PA, one college shared its student outcomes data with local high school principals and 
superintendents, and another brought in community members to participate with faculty and 
staff in developing a strategic plan. In WA, one college presented its disaggregated student 
outcomes data and discussed the achievement gaps among student subgroups with community 
leaders. 



Most colleges did not disseminate the results of internal data analysis to external 
constituencies. A possible explanation for this hesitancy came from a senior administrator who 
was involved in discussing some of her college’s disaggregated student outcomes data: 

The president wants anything we do in the external community to make the 
college look good, not to let the community know all of these things we 
found out with our data about how badly we are doing, what our gaps are 
with race/ethnicity. 
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Comparison of PA and WA Colleges with 
Round 1 Achieving the Dream Colleges 

This section compares the progress of the PA and WA colleges in implementing the 
first three steps of the Achieving the Dream improvement process with that of the Round 1 
colleges at the same point (a year-and-a-half) in the process. 

Leadership commitment 

College leaders at both the Round 1 Achieving the Dream colleges and the PA and WA 
colleges expressed a commitment to Achieving the Dream goals and values and viewed the 
initiative as consistent with college goals and priorities. College presidents and senior 
administrators at the PA and WA colleges were more likely than their peers at Round 1 
institutions to view Achieving the Dream as helping their college comply with regional 
accreditation standards. WA colleges were more likely to view Achieving the Dream as aligned 
with state policy than either the Round 1 or the PA colleges. 

College leaders demonstrated their commitment by reallocating resources in support of 
initiative efforts — most notably WA and PA presidents who used initiative and college funds 
to build IR capacity — and designated senior administrators to lead the initiative. To an even 
greater extent than did the Round 1 college leaders, the PA and WA presidents delegated 
oversight of the initiative to respected senior administrators rather than to project managers or 
other administrators outside of college leadership. 

The effects of leadership turnover on the commitment to the initiative were less of a 
concern among the PA and WA colleges than among the Round 1 colleges. Only 3 of the 13 PA 
and WA colleges had recently experienced a turnover in college leadership compared with 9 of 
the 27 Round 1 colleges. Furthermore, in each of the three PA and WA colleges that 
experienced turnover, the new president expressed full support for the initiative. As with the 
Round 1 colleges, the research team found that stable leadership of initiative teams by senior 
administrators helped with initiative planning and implementation. 

Faculty and staff receptiveness to the initiative 

Faculty and staff at both the Round 1 colleges and the PA and WA colleges were 
mostly supportive of Achieving the Dream goals and principles. Yet, college personnel at a 
handful of the PA and WA colleges and the Round 1 cohort of schools suggested that some of 
their colleagues were concerned that the initiative encourages strategies targeted toward certain 
groups of students. 

As with the Round 1 colleges, the PA and WA colleges presented the results of their 
data analysis to faculty and student services staff. The data came as a surprise to many college 
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personnel in both the Round 1 and the PA and WA colleges. Faculty and staff at the PA and 
WA colleges appeared more likely than those in the first-round colleges to view the data 
showing poor student performance or achievement gaps as motivation to improve. The PA and 
WA faculty and staff were less likely than their peers at Round 1 colleges to describe poor 
student performance as the result inadequate prior student preparation, or to blame their 
colleagues for the lack of student success. Faculty and staff at half of the Round 1 colleges 
expressed concern that data on student performance would be used to penalize them. The 
research team did not hear a similar concern from faculty and staff at the PA and WA colleges, 
although college personnel at several of the third-round colleges did echo comments heard by 
the Round 1 research team that a focus on student retention could undermine educational 
quality. 



Process for Identifying Student Achievement Gaps 
and Designing Strategies 

The PA and WA colleges relied to a greater degree than the Round 1 colleges on an 
analysis of their own college’s data in identifying gaps in student achievement. While only 
about half of the Round 1 colleges used longitudinal cohort tracking as part of their analysis of 
student performance, all but one of the PA and WA colleges did so at least to some extent. 
Moreover, to a greater extent than the Round 1 colleges, the PA and WA colleges used their 
own data to choose improvement strategies, rather than selecting the strategies to implement 
before the data analysis was completed, as was the case with many of the Round 1 colleges. 

About half of the WA and PA colleges successfully engaged faculty on a fairly wide 
scale in the process of designing strategies, a proportion comparable to that for the Round 1 
colleges. Few colleges in the WA, PA, or the Round 1 Achieving the Dream cohort actively 
engaged adjunct instructors in the planning process. And, as with the Round 1 colleges, the PA 
and WA colleges had only limited involvement of students and community members in the 
initiative. 



Summary 

The research team found widespread support across the 13 PA and WA colleges for the 
Achieving the Dream goals and principles. The commitment of senior leadership to improving 
student success, particularly among disadvantaged students, was evident among the colleges in 
both states. College presidents and senior administrators viewed Achieving the Dream’s focus 
on using data to improve student outcomes as consistent with trends in both accreditation 
standards and state policy. As a result, almost all of the PA and WA presidents were strong 
advocates for the initiative on their campuses and were actively engaged in Achieving the 
Dream efforts. College presidents and senior administrators demonstrated a willingness to re- 
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allocate college resources to support initiative activities, and many suggested that the initiative 
will serve as a framework for current and future student success efforts. 

Faculty and staff in both states generally had a favorable view of the initiative, yet some 
were worried about the time commitment, while others feared that the focus on student 
progression could lead to a lowering of academic standards at their institutions. At a few 
colleges, some personnel also expressed concern that their college had developed strategies for 
specific groups of traditionally disadvantaged students, rather than programs that touched all 
students. 

Achieving the Dream calls for colleges to use data on student progression to identify 
gaps in student achievement. As with the Round 1 colleges, limited IR capacity was an obstacle 
for many PA and WA colleges as they tried to identify areas of poor student outcomes. Despite 
this challenge, all 13 colleges used an analysis of their college’s data as the primary means of 
identifying gaps in student achievement. The colleges used both qualitative and quantitative 
data to identify and prioritize problems areas. Most of the colleges used the Achieving the 
Dream database to help identify problems areas, and all but one college also used student focus 
groups or student surveys. 

Seven of the 13 colleges engaged faculty and staff on a fairly wide scale in the process 
of using data to develop improvement strategies. At the other 6, only a relatively small number 
of faculty and staff were actively involved in analyzing data and identifying strategies for 
improvement. There is room for improvement in faculty and staff engagement at all of the 
colleges moving forward. With a few exceptions, the PA and WA colleges used the analysis of 
college data to guide the development of the strategies, though national research on student 
success interventions and lessons learned from student success efforts elsewhere were also used 
to design strategies. While college personnel were generally committed to the initiative, 
turnover among presidents, senior administrators, and institutional research staff delayed 
progress on the initiative at a handful of colleges. 
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Chapter 5 

Strategies for Improving Student Success 

Introduction 

Achieving the Dream encourages colleges to implement systemic interventions that will 
have a significant impact on student performance, rather than “boutique” programs that will 
benefit small numbers of students. Thus, after identifying barriers to student success and 
designing improvement strategies during the planning year. Achieving the Dream colleges are 
expected to move on to step 4 of the initiative’s institutional improvement process: they are to 
begin implementing their strategies, to evaluate the outcomes of their strategies, and to use the 
results to make further improvements and scale up those that are successful. 

This chapter describes the Achieving the Dream strategies being implemented by the 13 
PA and WA colleges. It discusses the progress that colleges were making in the first year of the 
four-year implementation period and identifies several key factors that influenced the progress 
of strategy implementation. The chapter also indicates how far along the colleges were in 
evaluating the outcomes of the strategies, as well as in implementing their plans for scaling up 
successful interventions. It concludes by comparing both the nature of the strategies 
implemented by the PA and WA colleges with the Round 1 Achieving the Dream colleges and 
the progress of the colleges in implementing them at a similar phase of involvement with the 
initiative. 

The description and analysis of the colleges’ student success strategies are based on 
field visits to the PA and WA colleges during spring 2008. Information was also obtained from 
the colleges’ implementation plans, entries made by the colleges in an online Achieving the 
Dream database in late spring 2008, and reports by data facilitators working with these colleges. 

Overview of Colleges’ Strategies 

Each PA and WA college developed an implementation plan during the 2006-07 
planning year which described college strategies for breaking down barriers to student success 
and indicated which student populations to target and the scale of the intervention. The 
Achieving the Dream initiative provides colleges with substantial freedom in identifying and 
designing institutionally relevant student success interventions. Colleges may decide to 
implement new programmatic strategies such as a learning community or a student success 
course, or they may expand or improve existing programs. Colleges may also choose to modify 
college policies, such as restricting late registration or mandating academic advising for 
developmental education students. 
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As described in the previous chapter, colleges analyzed student outcomes data and 
many conducted student and faculty focus groups prior to developing their strategies. However, 
the strategies were also clearly informed by resources from the Achieving the Dream website 
and Strategy Institutes, a review of the literature on effective practices, and lessons learned from 
past and current student success efforts at the colleges. The types of strategies implemented by 
the PA and WA colleges fall into the following seven broad categories: 

• Advising. 

• Developmental education. 

• Financial support. 

• First-year experience. 

• High school and community outreach. 

• Professional development. 

• Supplemental instruction, tutoring, and study groups. 

The types of strategies implemented by the colleges and the number of colleges that 
implemented each strategy are summarized in Table 5.1. 
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Achieving the Dream: Community Colleges Count 

Table 5.1 



Strategies Implemented at Round 3 Colleges as of Spring 2008 
by Type and Frequency 



Category/Strategy 


Description 


*Number 

of 

Colleges 


Advising 


Helps to keep students academically on track. 


10 


Early Alert System 


Identifies students at risk based on poor attendance and 
academic performance, connecting them with services such 
as counseling, tutoring, financial aid, etc. 


2 


Placement Testing 


Provides advising, particularly to first-time students, with 
regard to appropriate course selection based on placement 
testing to determine basic skill levels. 


3 


Enhanced Student 
Advising 


Provides expanded advising services to at-risk students in an 
effort to curb attrition rates and promote academic and 
personal success. 


3 


Mentoring 


Offers faculty and/or staff personal guidance to students, 
through one-on-one or group efforts, in an effort to help 
them reach their educational goals. 


1 


Restricting Late 
Enrollment 


Disallows late registration for all students in order to 
increase student success. 


1 


Develonmental Education 


Addresses achievement gaps for students in 
developmental education and increases the number of 
students moving on to college-level classes. 


18 


Academic Policy 


Modifies existing developmental education policy to 
increase the number of students who are able to transition 
from developmental education to for-credit courses. 


1 


Curriculum Redesign 


Improves student learning outcomes by revising or 
restructuring existing courses and practices. 


1 


Instructional Software 


Offers computer-based developmental education software as 
a learning tool. 


1 
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Category/Strategy 


Description 


*Number 

of 

Colleges 


Develoomental Education 

(continued) 






Instractor/Cohort-B ased 
Learning 


Offers support similar to the learning community, with the 
expectation that students will be tracked and will have the 
same instructor for at least two or more courses. 


1 


Learning Communities 


Provides support to developmental education students 
through enrollment in a community of students who take at 
least one developmental course and another course together 
as a group. 


4 


Multiple Developmental 
Education Placements 
Support 


Focuses on orientation and advising for developmental 
education students who place three levels below college- 
level math and at least one level below in reading or writing. 


1 


Summer Prep Programs 


Provides accelerated academic support, often in the form of 
intensive summer programs, to students who placed in 
developmental education. 


1 


Course Expansion 
/Restructuring in Math 


Adds new courses, adds content to existing courses, or 
modifies course content in order to prepare developmental 
education students for success at the next level. 


7 


Transition from Adult 
Basic Skills to College 


Provides bridge programs and other interventions designed 
to assist students in transitioning from non-credit adult basic 
education courses to college-level degree and certificate 
programs. 


1 


Financial SuDDort 
Strategies 


Assists struggling students with monetary and indirect 
supports. 


1 


Einancial Aid 


Provides direct or indirect financial support to students in 
order to encourage persistence and academic success. 


1 


First-Year Experience 


Helps students begin college with the tools they need to 
succeed. 


8 


Student Success Courses 


Increases beginning students’ knowledge of how to navigate 
the college in order to become efficient and productive 
learners. 


6 


New Student Orientation 


Provides individual and group orientation to first-year 
students. 


1 
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Category/Strategy 


Description 


*Number 

of 

Colleges 


First- Year Experience 

(continued) 






Academic Progress 
Reports 


Offers regular and consistent communication to first-time 
students about their academic progress in courses. 


1 


High School and 
Community Outreach 


Conducts outreach to high school students and 
community members to address the increasing number 
of applicants unprepared or under prepared for college- 
level work. 


1 


Placement Testing in the 
High Schools 


Provides high school students and their teachers an 
opportunity for students to be evaluated on college 
preparedness before graduation. 


1 


Professional Develonment 


Provides opportunities for faculty, staff, and 
administration to attend conferences, trainings, 
workshops, and lectures focused on ways to improve 
student success and the learning climate on campus. 


4 


Professional Development 
Programs 


Provides opportunities for faculty, staff, and administration 
to increase their personal and/or professional expertise in 
such a manner that the experience will improve student 
achievement. 


4 


SuDDlemental Instruction/ 
Tutoring/Studv Groups 


Provides access to assistance with instructional 
content outside of the classroom. 


8 


Conferencing 


Meets in or out of the classroom to provide intensive faculty- 
student consultations and academic support services. 


1 


Tutoring Support 


Provides intensive reinforcement to individual students or 
groups in the classroom or outside of it. 


2 


Supplemental Instruction 


Provides additional teaching and/or tutoring as a course to 
students in or outside of the classroom. 


4 


Study Groups 


Encourages groups of students with similar academic 
strengths or deficiencies to meet and provide one another 
with peer support and tutoring. 


1 



*Some Round 3 colleges have implemented more than one type of strategy per category. 
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Prevalent Achieving the Dream Strategies 
Implemented by the Colleges 

This section describes in detail the four most prevalent categories of strategies 
implemented by the PA and WA colleges: (1) developmental education interventions; (2) 
supplemental instmction, tutoring, and study groups; (3) student advising; and 4) first-year 
experience strategies. 

Developmental Education Strategies 

A majority of incoming PA and WA students required remediation in at least one 
content area, and many students required developmental coursework in multiple areas. College 
personnel across both states described serious challenges to serving students who arrived on 
campus unprepared for college-level work. As with their peers nationwide, such students often 
struggled to complete their colleges’ developmental education sequence and transition to and 
succeed in college-level courses. Because of the low student success rates of underprepared 
students, developmental education received considerable attention as the Achieving the Dream 
colleges began developing and implementing initiative strategies. For some PA and WA 
colleges, developmental education had long been recognized as an area of concern, and faculty 
and staff already had in mind interventions that they wanted to implement. Yet, for most 
colleges, the analysis of student outcomes data during the planning year, combined with student 
and faculty focus groups, helped guide college personnel to particular interventions. 
Administrators, faculty, and staff at the PA and WA colleges identified several problem areas 
within developmental education, including the following: 

• Inaccurate placement, 

• Students who placed into developmental education but delayed taking 
developmental courses until the end of their academic careers, if at all, 

• Inadequate academic and counseling support, and 

• Poor student performance and high dropout rates. 

Twelve of the 13 colleges implemented at least one strategy that targeted students in 
developmental education courses. Presented in Table 5.1, they involved the modification of 
academic policies, including the way that students were placed into developmental education; 
cohort-based learning and learning communities; curriculum restructuring; and course revision 
and expansion, particularly in developmental math. 

A few colleges revised the processes by which new students were placed into 
developmental coursework. In PA, faculty at one college suggested that the college had 
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historically done little to understand problem areas in developmental education. Upon joining 
the initiative, a group of faculty began analyzing the college’s ACCUPLACER cut scores, the 
sequence of developmental education courses, and student outcomes. The college realized that 
students placing into developmental courses were not taking the courses until the end of their 
academic programs, if at all. As a result of this analysis, the college began requiring students to 
take the developmental education sequence when they first enrolled at the college. The college 
also set new cut scores and reorganized both course content and the sequence of math courses. 
It was planning to evaluate these efforts to determine if they resulted in improved student 
outcomes. 

Student success in developmental math was a particular concern for several PA and 
WA colleges; 11 of the 13 colleges pursued strategies that targeted students who placed into 
developmental math. At one PA college, where approximately 90 percent of incoming students 
placed into developmental math, the data collection and analysis process during the planning 
year led to a focus on those students. According to an IR staff member at the college 
“Everywhere we looked pointed to developmental math.” The college created a developmental 
education taskforce that used cohort tracking of developmental math students and focus groups 
with both students and math faculty to identify specific problem areas with developmental 
math. As a result of this process, the college began testing a different developmental education 
placement exam, the ACCUPEACER. The college was also considering adding supplemental 
instruction to the existing developmental math courses. The taskforce generated considerable 
enthusiasm among a core group of faculty and staff and was motivating them to stay involved in 
implementing and revising improvement strategies for developmental education students. 

Seven of the Round 1 Achieving the Dream colleges worked with local high schools to 
increase college readiness as part of their initiative strategies. Among the PA and WA colleges, 
however, only one college adopted this strategy (although others were doing so apart from 
Achieving the Dream). That college had made developmental math the number one priority of 
its Achieving the Dream efforts, had begun working with local high school faculty to improve 
the math readiness of high school graduates, and planned to offer math placement tests for high 
school students. In addition to outreach to area high schools, the college began implementing 
supplemental instmction in math courses; developed and offered an intensive summer math 
course, and implemented online math tutoring. At the time of the research team’s spring 2008 
visit, the college was looking to hire a faculty member dedicated solely to teaching 
developmental math. 
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Profile of a Developmental Education Strategy: 
Multiple Developmental Education Placements Support 



One WA college sought to increase student retention through a one-on-one advising- 
centered strategy for underprepared students. The college mined its student outcomes data to 
identify a group of students who were most at risk: those who placed three levels below 
readiness for college-level math and at least one level below in either reading and/or writing. 
Termed the MP3- 11 initiative (reflecting the students’ relative placement scores), strategies 
entailed development of a learning community that included intensive one-on-one 
mentoring/advising, as well as support for college writing and math with the goal of increasing 
retention rates by 10 percent. Partially implemented, the MP3- 11 initiative may be one of the 
most ambitious and challenging strategies implemented as part of Round 3 of Achieving the 
Dream, primarily because of its highly at-risk target population. The college was paying 
adjuncts to participate in the extensive student support systems required as part of the strategy. 
As a new phase of the intervention, writing faculty were embedding mentoring/advising in a 
pre-college writing course that would be linked to developmental math as part of the learning 
community. The strategy required that writing, math, and reading faculty collaborate in the 
identification of potential students for MP3- 11 and in the future planning and scaling up of 
appropriate and successful interventions. Participating adjunct faculty were paid to participate in 
the meeting, planning, and mentoring activities supporting the strategy. 



Supplemental Instruction, Tutoring, and Study Groups 

Eight of the 13 PA and WA colleges developed strategies for providing students with 
additional learning support resources to help them master course content. Four of them 
implemented supplemental instruction in which peer leaders were h ir ed by the college to attend 
classes and schedule review sessions for students. Two colleges were working to expand their 
tutoring services. One of them was expanding its online tutoring capacity to reach students who 
lived a considerable distance from the campus and the other was experimenting with 
“embedded tutoring,” in which a peer tutor shadowed struggling students in their courses each 
day, observing, taking notes, and then helping them during after-class hours. 

The supplemental instruction, tutoring, and study group strategies were often targeted to 
developmental education students or students in gatekeeper courses. Several colleges were 
considering or beginning to make supplemental instruction and tutoring mandatory in certain 
developmental math courses, which colleges generally found to be a barrier for many students. 
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Profile of a Supplemental Instruction Strategy: 
The Study Club as a Community and Academic Tool 



An Intermediate Algebra Study club was created by one WA college to target students 
who needed additional academic assistance in Intermediate Algebra, a designated “gatekeeper” 
course at the college. Although other interventions had been tried to increase student persistence 
in math, particularly for African-American males, administrators learned that students studying 
Intermediate Algebra frequented the math lab in the greatest numbers and decided to link this 
desire for academic assistance to peer support and community. Students and faculty posted 
flyers around the campus and in all three of the tutoring centers announcing the formation of an 
Intermediate Math study club, with its own math lab faculty facilitator and peer tutors. Math 
faculty also informed students in each Intermediate Algebra course about the study club. The 
study club allowed students studying Intermediate Algebra to work with and support one 
another, while also helping them network and connect with supports. The college’s 
implementation of this strategy was in the early phase, but the college will attempt to compare 
the success rates of students who attended the Intermediate Algebra Study Club with those of 
the Intermediate Algebra students who did not attend the club with the hope of increasing the 
pass rate for Intermediate Algebra by 5 percent over the next two years. 



First-Year Experience: Student Success Courses and 
Academic Progress Reports 

A third category of strategies focused on the student experience during the first year of 
college. Research suggests that positive academic and personal experiences during the first 
semester and year of college are critical to student persistence and success. Yet, community 
colleges generally stmggle to successfully integrate new students into the college environment. 
Many incoming community and technical college students have little understanding or mastery 
of the skills that are needed to succeed in college or of how to navigate the college environment. 
Often they are “nontraditional” students — first-generation college students, heads of 
households, full-time workers, or caregivers of parents and children — who stmggle to balance 
life demands with class schedules. The implementation of new student orientations and student 
success courses were two strategies used by Round 1 colleges to engage these students early in 
their college careers and improve student outcomes. 

Student success courses, geared toward providing first-year students with the 
knowledge and skills they need to succeed at college, comprised a prevalent strategy among PA 
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and WA colleges. While only one WA college followed the example of Round 1 colleges and 
revised student orientation, six of the PA and WA colleges either created or revised the content 
and delivery of college success courses. The courses included seminars and workshops on a 
variety of topics, such as time and financial management and foundational skills that directly 
impact academic success, such as note taking and how to study. Practical, college-specific 
knowledge on financial aid, how to apply for a scholarship, or where to go in the college with 
personal concerns was also typically included. For example, a college in WA revised a 
mandatory student success course for developmental education students and students with an 
undeclared major. This revised course, designed to help these students acclimate to the college 
environment and refine their study skills, was widely praised by college personnel. One 
administrator said: “The [student success course] has been phenomenal. It’s a result of 
Achieving the Dream.... Our mission here is education for the masses. A lot of times those 
folks don’t have a basis for higher education in their families. The [student success course] has 
filled in that gap.” 

Similarly, a PA college developed and implemented a college success course composed 
of three modules that could either be offered separately or combined to create a three-credit- 
hour course. One module focused on what students need to do to successfully navigate the 
college. A second module was designed to help students choose the most appropriate academic 
major. The third focused on steps required to transfer to a four-year college. 



Profile of a First-Year Experience Strategy: 

Academic Progress Reports as a Way to Promote Persistence 

One WA college offered academic progress reports to improve persistence. Students 
received descriptions of expected learning outcomes of the program early in their first semester 
and regular feedback on course performance. Faculty and staff were in regular conversation 
with first-time students, documenting their progress while also providing some advising about 
resources available on the campus or about personal and academic concerns that might prevent 
a student from completing the course. This early and frequent conversation was contextualized 
to occur in courses within the following program areas: Auto Body Repair and Refinishing, 
Culinary Arts, Early Childhood Careers, and Pharmacy Technician. The strategy was piloted in 
fall 2007 with faculty members and counselors working together to meet the needs of new 
students in each of the programs. This intervention was in the process of being evaluated, but 
administrators expected to see a 10 percent decrease in the number of students who withdrew 
from the programs that received the intervention. 
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Advising Strategies 

Eight of the PA and WA colleges implemented at least one new advising strategy. As 
was the case with the student success courses, colleges viewed advising as a key activity to help 
students navigate the college environment and fulfill their personal and academic goals. Some 
of the strategies pursued by the PA and WA colleges reflected novel approaches to advising. 
Several colleges were targeting underrepresented students for enhanced or expanded student 
advising, including first-time college students, Hispanic students, ESL students, and 
academically underprepared students. Eor example, a WA college was providing advising to 
students transitioning from non-credit courses to college-level degree and certificate programs. 
According to the president, the college’s most strategic goal focused on moving ESE students 
into career technical training and degree programs that would allow them to achieve a certain 
level of economic security. Eor these students the college developed “Pathfinder/bridge 
courses” in health care, business, education, and human services, as well as a generic version for 
students who were undecided or wanted to transfer. Advising was a key component of 
interventions that the college hoped would support the successful move of more students into 
credit-bearing programs. 

An early academic intervention proposed at another WA college targeted Opportunity 
Grant students near the beginning of their first semester. Opportunity Grants were need-based 
financial aid provided by Washington State to low-income students who were often considered 
to be at risk due to socioeconomic status and other factors, such as first-time and first-generation 
student status. Administrators and faculty cited research indicating a high probability that these 
students would drop out before earning a certificate or degree. The students were encouraged to 
initiate study groups and to seek tutoring as early in the semester as possible. Eaculty members 
were required to submit quarterly grade checks to determine if extra help was needed. 

Several colleges began considering mandatory advising policies, reflecting a larger 
trend across the WA and PA colleges toward more prescriptive academic and student services 
policies. Eor example, one WA college was dramatically changing the scope and target student 
population of its advising program, and was particularly focused on targeting developmental 
education students and first-time college students. The college’s goal for its advising strategy 
was to require advising for the following students: (1) full-time and/or degree students, until 
they accumulated 30 credits; (2) transfer students, until post-transfer for one quarter; and (3) 
students who changed their major/program, for at least one quarter. Similarly, a college in PA 
was restricting late registration to prevent students from registering after the first day of class. 
An administrator from this PA college explained the reason for the new policy: “As a 
community college we were proud of the fact that we were open-admission and people have the 
right to fail. Now we’re saying that you don’t have the right to fail. We will be more 
prescriptive.” 
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Profile of an Advising Strategy: 
Mandatory Advisement 



At one WA college student focus groups and CCSSE results revealed that students 
needed additional help in understanding the college and wanted better advising. The college 
was making advising mandatory for students who test into developmental courses, revising the 
intake interview process, and creating an assessment and educational planning session. The 
college made progress in establishing new procedures for advising and provided training and an 
advising manual to faculty who said they were unprepared to adequately advise students. Some 
faculty members received advisor training during the faculty in-services in the middle of the 
first implementation year. Counselors were expected to provide advising training sessions for 
all faculty in fall 2008 using both Achieving the Dream and Title V grant funds. 



Colleges’ Progress in Implementing Strategies 

Toward the end of the first implementation year, in spring 2008, the research team 
evaluated the progress of strategy implementation. At that time, all of the colleges had begun at 
least preliminary implementation of at least one strategy as part of Achieving the Dream. The 
progress on implementation is discussed below, with strategy implementation categorized into 
three levels: under development, partial implementation, and full implementation. 

Strategies Under Development 

For most colleges, the analysis during the planning year revealed poor student outcomes 
overall and noticeable achievement gaps, but it produced no obvious or quick-fix solutions. By 
spring 2008 many colleges were still grappling with how to use their limited resources to design 
and implement strategies that best met the needs of their students. For these colleges, the 
planning of initiative strategies extended into the implementation year, with most strategies still 
mostly in the planning stage. 

Four of the 13 colleges were still in this early implementation phase; the colleges had 
staff working on the strategies and were in the process of making preliminary steps toward 
implementation, but the majority of their strategies were still under development. Colleges with 
strategies in this early implementation phase often expressed a need for additional research and 
planning time. Other colleges were reviewing potential changes in institutional policies. Several 
college strategies required additional training for staff involved. For example, one college was 
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still preparing a strategy linking and sequencing developmental English and math courses with a 
three-hour freshman seminar. The college’s progress with implementing this strategy stalled 
because of turnover among the project leadership. “Our Achieving the Dream efforts started off 
and stumbled. This fall [2008] really is our kick-off.” Another college was creating a mentoring 
system around developmental math courses. At both of these institutions, extensive training and 
orientation for faculty was required before the strategy would begin to touch students. 

Partial Implementation 

Strategies were categorized as partially implemented if the colleges were still piloting 
them or were in the process of revising or modifying them. At 9 of the 13 colleges the majority 
of initiative strategies were partially implemented. The experience of a PA college that chose to 
revise a student success course that did not seem to be effective provides an example of a 
partially implemented strategy: Student outcomes data and feedback from focus groups 
suggested that a one-credit student success course for students of color did not appear to be 
benefiting students, and focus groups revealed that students were reluctant to take the course 
because they felt it stigmatized them. The college therefore decided to dedicate a section of the 
course — rather than the entire course — to Latino and African-American students. 

Full Implementation 

Strategies that had reached the college’s proposed scale and target population were 
categorized as fully implemented. While three PA and two WA colleges had at least one 
strategy that had been fully implemented by spring 2008, none of the colleges had a majority of 
their strategies fully implemented. The few strategies that had been fully implemented were 
generally those with which the colleges had some experience in the past, those that represented 
a change in college policy or procedures, or were professional development activities for faculty 
and staff. For example, four colleges introduced professional development activities and two 
fully implemented them. The fully-implemented activities were diversity training at a small, 
mral college and IR staff training on how to increase productivity at a college with limited 
resources and a growing demand for data. Each activity addressed needs and concerns specific 
to the institution and had a goal of addressing that pressing need in a short period of time. 



Factors Affecting Strategy Implementation 

The previous chapter described several factors that influenced college progress in 
identifying student achievement gaps and developing strategies for addressing priority 
problems. This section describes how several of those same factors were also key to college 
progress in the implementation of initiative strategies. In addition, this section explains how 
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collaboration between faculty and student services staff and efforts at recruitment of students 
into strategies affected the progress of colleges during the implementation year. 

Faculty Engagement as a Spur to Strategy Implementation 

Achieving the Dream encourages broad-based engagement of college personnel in 
implementing and assessing the effectiveness of strategies. Eight of the PA and WA colleges 
had successfully engaged faculty and staff in implementing initiative strategies. A core group of 
faculty at one PA college described a feeling of ownership of the initiative, which motivated the 
group to design and implement a key developmental education strategy. The college’s faculty 
had historically not been consulted about how students were placed into developmental 
education and had little experience evaluating the developmental education program. The 
college hired a new president just prior to joining the initiative, and faculty and staff reported 
that the president was dedicated to improving developmental education and was receptive to 
faculty and staff input. He allowed developmental education faculty to take ownership of 
strategy development and encouraged faculty buy-in and involvement. A reading instmctor 
said: “What’s important is that the college has clearly stated that this is a faculty-driven 
initiative. We present what we’ve been doing. They let us put this together. This has been a 
major plus.” According to a faculty member on the core team, a majority of the college’s faculty 
was involved in the implementation of at least one strategy: “Sixty-five percent of the [full- 
time] faculty are involved. I’m confident about this number. We just checked.” 

Even among colleges that successfully engaged faculty and staff, several had difficulty 
initially in recmiting faculty to implement strategies. At one college, few faculty and staff were 
showing up for professional development activities, one of the college’s strategies. The IR 
director of the college said the professional development sessions were “loosely mandated,” but 
few faculty were taking part: “People are not participating. In opening sessions at the beginning 
of the semester, it’s administrators and a few faculty who are living the message. Those faculty 
who are not there are in most need of the message.” At a second college, in-fighting between 
faculty members of one department had stalled implementation of one of the college’s 
strategies. 

Some college faculty and counselors considered participation in Achieving the Dream 
strategies as an add-on to their already full workday schedules. Eor example, a few college 
personnel were concerned that the initiative would be another temporary fad, and were thus 
reluctant to commit time and energy to the effort. As mentioned in the previous chapter, one PA 
college instructor indicated that he had seen many previous student success efforts fall by the 
wayside at that college after the initial excitement waned: 




Faculty are concerned about investing in initiatives. In the past when we have 
done so much with learning communities [and] active learning, then nothing 
changes. Faculty are afraid that we will invest all this energy that we don’t 
really have in Achieving the Dream. And then next year we won’t have the 
resources to follow through. 

The college’s recent history of several failed or unresponsive presidencies contributed 
to widespread cynicism. Another faculty member at the college echoed this sentiment: “We 
have limited time and energy, and I get the feeling that the faculty, who have been here a while, 
have tried many things, and seen things float away into the mist, and they are reluctant to begin 
again.” 



Other faculty and staff were committed to the initiative, but were simply stretched thin 
with their current job responsibilities. Because administrators, faculty, and staff at several 
colleges suggested that the initiative required too much time and effort, providing release time 
might facilitate increased participation. One faculty member, speaking about professional 
development activities, put it bluntly: “If you want faculty to do more, there needs to be a top- 
down incentive.” 

Student Services Staff Engagement 
as a Spur to Strategy Implementation 

At some of the colleges that were further along in strategy implementation, the colleges 
had engaged student services staff in student success efforts and had built strong collaboration 
between faculty and student services personnel. At 6 of the 13 colleges. Achieving the Dream 
substantially increased student services involvement in student success efforts, and at 6 colleges 
the initiative strengthened collaboration between faculty and student services. Several colleges 
were making progress in breaking down divisional silos. Student affairs personnel at one 
college, for example, discussed the improved communication between faculty and student 
services in revising their advising program for developmental education students: 

Achieving the Dream as an initiative has buy-in across campus. Faculty have 
a greater understanding of the student affairs role as a result of their 
involvement in the respective interventions. It has broken down some of the 
barriers between academics and student affairs. 

A student affairs colleague at the same college added: “There is more willingness to 
pick up the telephone and ask questions rather than making assumptions which often turn out to 
be wrong.” Learning communities, which link developmental math and English courses, college 
success courses, and introductory college-level courses, were another type of strategy that 
required collaboration among a range of faculty and student services personnel. Four of the 13 
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colleges developed learning communities to help academically underprepared students. The 
colleges that had good collaboration across the various academic and student affairs divisions 
had made greater progress in implementing their learning communities. 

At a few colleges, inadequate collaboration between faculty and student services staff 
negatively impacted implementation, particularly across strategies that required personnel from 
various academic and student services departments to work together. At one college an 
administrator described the lack of collaboration between faculty and counselors who were 
jointly responsible for advising students: 

We don’t have that sense of community. I don’t know if it’s an “us against 
them” or lack of time. They don’t communicate across lines. I think a lot of 
time change will happen and the information won’t get to the advisors. They 
need to have the information before advising begins. So, in advising I don’t 
think they step out of their areas of comfort and seek out information from 
the departments. I don’t see that happening. 

Delayed Strategy Implementation Resulting from Personnel Turnover 

Considerable turnover in key personnel, a factor that delayed the collection and data 
analysis for some colleges during the planning year, also delayed strategy implementation at 
three colleges. In the most extreme case, turnover among college administrators involved in the 
initiative meant that the college had to essentially start again from the beginning of the planning 
stage during the first implementation year, having piloted only one or two strategies on a very 
limited scale. An administrator at this college explained: “It started off really well, then staff left 
and it all fell apart. People haven’t done as much since that time.” 

Colleges’ Difficulty in Recruiting Students into Initiative Strategies 

At least three colleges reported difficulty recmiting students into their strategies. For 
example, a PA college delayed the implementation of three learning communities planned for 
spring 2008 because there was insufficient student enrollment. The same college developed a 
two-week math “boot camp” — a short, intensive course focused on basics (mostly arithmetic) 
that students could take between or just prior to a regular semester to help them place into a 
higher-level math course. The intensive math course was offered at no cost to students during 
the summer, yet it failed to generate much interest from developmental math students. An 
administrator at the college said 15 students registered for the course, but several never showed 
up and others dropped it. Similarly, another college faced challenges recruiting students to 
participate in key advising and tutoring strategy sessions that were originally planned for 
outside of regularly scheduled class time. A faculty member said: 
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Students have been very unresponsive to incentives — we offered them 
priority registration, a bookstore gift certificate, free pizza for a focus group 
with a gift card. They say “yes we’d come.” And then only one person 
showed up. We keep trying to find out what is going on. Our sample size is 
so small. But we really don’t know why they aren’t responding. That is part 
of what motivated the decision to move to in-class interventions. 

College officials hypothesized that work and family responsibilities interfered with the 
ability of students to engage in out-of-class support systems. To help ensure participation, one- 
on-advising, tutoring, and mentoring were built directly into the in-class portion of the learning 
communities strategy. 



Evaluation of Strategies 

Several colleges had begun at least preliminary evaluation of their strategies. Yet, 
because many of the colleges faced delays in implementing their strategies, they generally had 
few evaluation results by the time of the research team visits in spring 2008 . 

Five of the colleges had formal plans for evaluating their strategies, but only two had 
developed what the research team considered to be sound evaluation designs. 

Several colleges had little prior experience in evaluating program outcomes, and they 
often lacked institutional research capacity to conduct high-quality evaluations of the strategies. 
At just over half of the colleges, overburdened IR staff and turnover among IR personnel 
hindered the evaluation of strategies. Weak collaboration between IR and faculty/staff was also 
an issue, with several colleges piloting interventions without much thought about proper 
research design. Bringing faculty and IR staff together at the “front end” would have had 
several potential benefits, including: facilitating IR-faculty collaboration toward more 
sophisticated program evaluation; encouraging faculty to think about program evaluation; 
increasing the likelihood that the evaluation is actually carried out; and decreasing the number 
of poorly thought-out data requests from instructors. 

Poor relations between administration and faculty also hindered efforts to evaluate 
strategies. At one college, tension between the administration (including IR) and faculty 
resulted in a math intervention with inadequate evaluation planning — essentially the math 
department assumed responsibility for the evaluation. The IR director was uncertain about the 
evaluation design and had not seen any results. 
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Scope of Targeted Population for Strategies 

The Achieving the Dream strategies being implemented at about half of the colleges 
tended to still be in the early pilot stages, affecting a relatively small group of students thus far. 
Some faculty from at least one college questioned the amount of resources spent on the 
initiative, considering the relatively few students touched up to that point. One faculty member 
said, “The numbers have been really small for the amount of effort that has been expended. I 
have real concerns about what’s going to come out of this given the small numbers.” 

Seven of the colleges had at least one or two strategies that were reaching large 
numbers of students. For example, one PA college was implementing the following strategies: 
requiring all entering students to take a placement test; aligning developmental education, 
gatekeeper math, and English curricula; and ending late registration. Faculty at another PA 
college reviewed research that showed students who registered late tended to have high failure 
rates, which led to a campus-wide policy to no longer allow late registration after the first day of 
class. 



Plans for Scaling Up Strategies 

A risk that the colleges faced was that Achieving the Dream-supported activities would 
cease once the grant funding runs out. Several colleges were under financial pressures or lacked 
discretionary funds, raising the question about the sustainability of their strategies. The current 
economic downturn means that there are likely to be further reductions, perhaps severe in some 
cases, in state spending for community and technical colleges. College leaders and senior 
administrators may hesitate to devote long-term funds to initiative-driven programs for fear they 
might not be able to continue funding after the Achieving the Dream grant expires. Yet, scaling 
up the successful strategies would be difficult without additional resources. One president said: 

Money from Achieving the Dream is a pittance. We would love to get Title 
III to make this sustainable and take it to scale. What’s missing in our 
program: to put a body to a body, peer support, faculty, and staff advisors, 
etc. We need dollars for human resources. 

With a handful of exceptions, few of the colleges had given much thought to bringing 
successful strategies to scale. Most colleges were still in the planning and early implementation 
phase and were experimenting with small-scale strategies to see what worked. The vice 
president of instmction at a WA college explained: 
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My impression is that we have a planning year and four years of 
implementation. For the first two years we are going to test and experiment 
with a wide variety of approaches. In the third year we will look at what we 
want to institutionalize, and test it. And then by the fourth year we would be 
in a position to know what we want to do. That seems to be consistent with 
what other colleges are doing, based on feedback from the Strategy Institute. 

The exceptions were generally the colleges that already had experience with a particular 
strategy prior to Achieving the Dream and had in place a supportive infrastmcture. One WA 
college was scaling up its college success skills course after finding improved outcomes for 
students who took it. The college compared the term-to-term retention and success rates of first- 
time students who completed the course with those who did not complete it and found a 20 
percent higher persistence rate for those who completed the course. Because of this finding, the 
number of sections of college success skills courses were to be increased in fall 2008, and 
students considered to be most at-risk would be required to enroll in the course. Yet, faculty and 
staff at the college reported being already stretched thin with their workloads. 

The experience of one large urban college in PA that proposed requiring all incoming 
students to take a student success course suggests the difficulties that can arise from campus- 
wide interventions that lack adequate planning and resources. A student services staff member 
at the college said: 

Too many students don’t know how to negotiate the college. We looked at 
requiring them to take a one-credit college survival course. We didn’t think it 
through. We weren’t prepared to implement a policy where we didn’t have 
enough faculty, enough sections. What were the penalties? We decided to go 
back to the drawing board so we can better identify the students who can best 
benefit from that course. 

Furthermore, while a few colleges had begun thinking about scaling up successful 
strategies, only two colleges appeared to have a plan for doing so. At one PA college the 
provost made sure that during the design phase the college thought about how to scale up each 
of its strategies, including the budgetary implications. She tried to not pilot a strategy that she 
could not scale up. For example, faculty wanted 20 percent release time to oversee students in 
supplemental instmction; she concluded that the college could not afford to do that over time. 
Instead, she found a graduate student to hire as a part-time supplemental instmction coordinator. 
A WA college was in the process of scaling up its advising strategy, yet the college was having 
difficulty. Its director of IE said the college was struggling to recmit volunteer advisors beyond 
the initial cohort that signed up to participate in the pilot intervention. 
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Comparison of PA and WA College Strategies with 
Round 1 College Strategies 

Types of Strategies Implemented 

The PA and WA colleges and the Round 1 cohort of Achieving the Dream colleges 
implemented similar strategies. College personnel at several of the PA and WA institutions 
suggested that the design of their strategies was influenced by the Achieving the Dream 
Strategy Institute, where they had the opportunity to learn from colleagues at colleges that 
joined Achieving the Dream in the first two rounds. 

The WA and PA college strategies were particularly focused on helping developmental 
education students complete their colleges’ developmental education sequence of courses and 
succeed in college-level courses. The Round 1 colleges focused many of their strategies on 
helping developmental education students as well. The Round 1 baseline implementation report 
found that strategies often reached developmental education students via course restructuring, 
learning communities, supplemental instmction, and intensive advising. Yet, the WA and PA 
colleges were focused to a greater degree than the first-round colleges on reforming the content 
of their developmental education courses and the sequence of developmental education courses; 
9 of the 13 PA and WA colleges focused their developmental education efforts in that area, 
compared to just 9 of the 27 Round 1 colleges. The WA and PA colleges were also more likely 
to focus their efforts on improving student performance in developmental math. All but one of 
the WA and PA colleges designed and implemented a developmental education strategy. 

Several strategies were found with similar frequency across the Round 1 and the WA 
and PA colleges. For example, college success courses were a popular strategy among the PA 
and WA colleges; 6 of the PA and WA colleges designed or revised a new college success 
course, as did 16 of the 27 Round 1 colleges. Eight Round 1 colleges implemented 
supplemental instmction, as did 4 of the PA and WA colleges. Eleven of the Round 1 colleges 
implemented learning communities, compared with 4 of the PA and WA colleges. Six Round 1 
colleges offered tutoring, compared with 2 of the PA and WA colleges. 

A few strategies were less prevalent among the PA and WA colleges. Eor example, 
early alert was more popular among Round 1 colleges as a means to reduce student attrition. 
Eight of the Round 1 colleges implemented early alert, compared with just two PA and WA 
colleges. Twenty-two Round 1 colleges reported using Achieving the Dream funds for 
professional development activities, compared with only four of the PA and WA colleges. 



94 




Progress in Implementing Strategies 

The PA and WA colleges and the Round 1 colleges had made similar progress in 
developing and implementing their strategies by the end of the initiative’s first implementation 
year. The Round 1 baseline implementation report found that the first cohort of Achieving the 
Dream colleges had made progress in developing strategies, but that the colleges still had a lot 
of work to do to implement their strategies and bring them to scale. The Round 1 research team 
wrote: 



The majority [of Round 1 colleges] can be characterized as either (1) having 
partially implemented their strategies, meaning implementation was 
occurring on a small scale, or (2) being in the early phases of 
implementation, meaning planning was still the main focus but staff had been 
dedicated to the effort and implementation plans were fully fleshed out. 

The research team’s visits to the PA and WA colleges found that few of the colleges’ 
strategies had been fully implemented by spring 2008. While five of the colleges had at least 
one strategy that had been fully implemented — the strategies had reached the college’s 
proposed scale and target population — most of the strategies at each college were either still in 
development or were only partially implemented. 

Evaluation and Scaling Up of Successful Strategies 

By the end of the first implementation year, both the PA and WA colleges and the 
Round 1 colleges had only minimal evaluation results from their strategies. Only about one 
fourth of the Round 1 colleges had developed plans for evaluating the effectiveness and impact 
of their strategies. Similarly, only four of the PA and WA colleges had designed formal 
evaluation plans at the time of the research team’s visit, with three of the colleges producing 
some early evaluation results from their strategies. 

The Round 1 baseline report suggested that the colleges still had a lot of work to do to 
bring strategies to scale. The Round 1 research team reported that “strategies that reach a large 
proportion of the student population appear to be the exception rather than the mle.” The PA 
and WA colleges were at a similar stage, with few of the colleges thinking seriously about 
bringing their strategies to scale, and only a couple having a solid plan to do so. 



'“Brock et al. (2007), pp. 96-97. 
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Summary 

The PA and WA colleges followed the Round 1 colleges in implementing a wide 
variety of student success strategies. Most PA and WA colleges focused their efforts on 
improving student outcomes in developmental education, and in developmental math in 
particular. While Round 1 colleges also designed strategies to help developmental education 
students, the WA and PA colleges were more focused on reforming both the content and 
sequence of their developmental education courses. 

The PA and WA colleges and the Round 1 colleges had made similar progress 
implementing their strategies. All of the PA and WA colleges had implemented at least one 
strategy at the time of the research team’s visit, but few had reached the planned scale and target 
population. Instead, most colleges had only partially implemented their strategies — they were 
either still piloting them or were in the process of revising and improving them. A couple of 
colleges in both PA and WA were still in the planning stage, with most of their strategies 
remaining largely under development. Turnover among college personnel and inadequate 
staffing were key factors in the delay of implementation at these colleges. 

Because many of the PA and WA colleges experienced delays in implementing their 
strategies, colleges generally had few evaluation results by the time of the research team’s visits 
in spring 2008. Moreover, few of them had formal plans for evaluating their strategies, and only 
two had developed what the research team considered to be sound designs for evaluating their 
strategies. At more than half of the colleges, overburdened IR staff and turnover among IR 
personnel hindered the evaluation of strategies. Weak collaboration between IR and faculty/staff 
was also an issue, with several colleges piloting interventions without much thought about 
proper research design. While a few colleges had begun thinking about scaling up successful 
strategies, only two colleges appeared to have a plan for doing so in order to reach substantial 
numbers of students. 
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Chapter 6 

Progress Toward Institutionalizing a Culture of Evidence 



This chapter assesses the extent to which the PA and WA colleges began to implement 
step 5 of the Achieving the Dream improvement process: institutionalize a culture of evidence 
on their campuses. It also compares the progress of the PA and WA colleges with that of the 
Round 1 colleges at a similar stage in the initiative. 



Analysis of the Colleges’ Progress 

The fieldwork at the PA and WA colleges was structured using the Achieving the 
Dream Framework for Improving Student Outcomes and Institutional Performance. The 
framework presents the initiative’s culture of evidence model of institutional effectiveness, 
described in the Introduction of this report. Following the visits to each college, the research 
team wrote a field report assessing the extent to which the college had implemented practices 
associated with the principles of initiative’s model. The team then rated each college and its 
practices using the tool presented in Appendix A, based on the framework. Note that the 
research team made its assessment based on the college’s status with respect to implementing 
policies and practices associated with the initiatives culture of evidence model, whether or not 
their efforts were the result of work on Achieving the Dream. 

Overall Assessment 

Tables C.l and C.2 in Appendix C show the detailed results from the research team’s 
ratings for the PA and WA colleges, respectively, using the tool we developed to measure 
implementation of the Achieving the Dream model of effective institutions (see Appendix A). 
Based on these ratings and associated field notes, the research team classified the colleges by 
their progress in institutionalizing a culture of evidence, as shown in Table 6.1. As of the time of 
our visits in spring 2008, the research team rated 2 of the 13 PA and WA colleges as having 
made little or no progress toward institutionalizing a culture of evidence; 3 as having made 
limited progress, although major obstacles remained; 4 as having made promising progress; and 
4 as having begun to institutionalize a culture of evidence on their campuses. 
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Achieving the Dream: Community Colleges Count 
Table 6.1 



Achieving the Dream Colleges in Pennsylvania and Washington State: 
Progress Toward Institutionalizing a Culture of Evidence 



Extent of Progress 


Pennsylvania 

Colleges 


Washington 

Colleges 


Little or no progress 


1 


1 


Limited progress, major obstacles remain 


1 


2 


Promising progress, some obstacles remain 


3 


1 


Beginning to institutionalize a culture of 
evidence 


2 


2 



Factors That Distinguish Leaders from Laggards 

There are a number of differences between the colleges that were making solid progress 
toward developing a culture of evidence and those that were not. They are discussed below and 
shown in Tables C.l and C.2. 

Leadership commitment to making necessary changes 

In colleges that were progressing, the president and other leaders not only said that they 
were committed to student outcomes, they acted on their convictions, showing a willingness to 
make substantive changes in institutional policy and practice and to make the investment of 
resources necessary to support such changes. For example, the president of one PA college 
established “process management” teams to review the college’s policies and procedures and 
identify changes that would promote student success. During the planning year, these teams 
examined both quantitative and qualitative data, researched promising practices, and made 
recommendations to the college’s administrative council. Based on the recommendations from 
these teams, the college established more consistent grading standards for faculty, revised the 
college’s developmental placement policies based on analysis of cut scores, and upgraded the 
course registration system to ensure that students followed policies. 
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Faculty and staff engagement 

One of the clearest differences is that colleges that were making progress toward 
building a culture of evidence were more effective in involving faculty and student services 
staff in efforts to improve student success. A PA college that made some of the most progress 
toward institutionalizing a culture of evidence organized its Achieving the Dream activities to 
comprise wide involvement by faculty and staff in improving student success. During the 
planning year, Achieving the Dream was introduced in the fall convocation and reinforced 
through in-service professional development activities throughout the year. As described in 
Chapter 4, the college established five work groups, each focused on one of the Achieving the 
Dream performance measures. Over 90 faculty and staff members from across the college 
participated in these groups, which analyzed data provided by the college’s institutional 
effectiveness staff and recommended strategies for implementation under Achieving the Dream. 

While that college was exemplary in its faculty and staff engagement efforts, the other 
colleges that were moving toward institutionalizing the Achieving the Dream culture of 
evidence model also made headway in engaging faculty and staff. In contrast, none of the five 
colleges that lagged in their efforts to build a culture of evidence made much progress in 
engaging faculty. Indeed, at four of them, top administrators seemed genuinely committed to 
the Achieving the Dream goals for improving success, but faced resistance from faculty. As 
discussed in Chapter 4, at one of these colleges the initial planning and implementation of 
Achieving the Dream was perceived as top-down, which caused substantial faculty opposition 
to the initiative that was still evident during the research team’s visit. At another college, with a 
history of conflict between administration and the faculty, the faculty members whom we 
interviewed indicated that they viewed any initiative coming from “the top” with suspicion. 
Some indicated that they saw Achieving the Dream as yet another effort by the administration 
to impose reform, saying that skeptics referred to the initiative as “Achieving the Daydream.” 
According to administrators, the coach and data facilitator encouraged the college to downplay 
Achieving the Dream as a separate new initiative and instead to emphasize the student success 
goals of the initiative in discussions with faculty and staff. 

Collaboration between faculty and student services staff 

Collaboration between faculty and student services staff on student success efforts was 
also stronger at the leader colleges. For example, the IR director at one WA leader college 
discussed how Achieving the Dream, by its explicit focus on barriers to student success, had 
improved collaboration across the institution: 
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Seeing those barriers and dealing with those barriers explicitly has been a 
result of Achieving the Dream and [has] moved the college forward in 
breaking down silos. Before, that happened on an individual basis, but not 
across the institution. Achieving the Dream has moved the process forward 
much faster. 

Laggard colleges, conversely, often struggled to overcome the “silos” between 
academic and student affairs that characterize many community colleges. For example, faculty 
leaders at one college rejected efforts by student services staff to offer an improved college 
success course. Even at leader colleges there seemed to be room for improvement on this front. 

Cross-division communication 

In general, communication across divisions seemed to be stronger at the leader colleges 
than at the laggards. Perhaps reflecting this difference, the leader colleges were more likely to 
have in place committees for bringing together personnel from across the institution to work on 
student success. For example, to ensure that the work of collecting and analyzing data on 
student success was institutionalized, one college converted its data team into a standing 
“institutional assessment council,” with responsibility to continue analyzing data on student 
success, reviewing results from evaluations (conducted according to a master evaluation plan 
established through Achieving the Dream) and making recommendations for improvement to 
the college’s planning council, which was the institutionalized version of the Achieving the 
Dream core team. Prior to joining the initiative, one WA college had established a “student 
outcomes commission,” comprised primarily of faculty, that used data to monitor student 
progress and the effectiveness of efforts to improve student success. The commission analyzed 
student outcomes even at the level of instmctor and section. When the college joined Achieving 
the Dream, the commission became more consistent in disaggregating data by student 
characteristics and as a result uncovered achievement gaps that were not recognized before. 
According to faculty and administrators we interviewed, the commission was widely respected 
at the college and had a lot of clout on decisions related to student success. 

Institutional research and information technology capacity 

The experience of the PA and WA colleges indicated that having strong IR capacity 
was helpful, but not sufficient, in building a culture of evidence. It was also essential to use the 
data collected as a basis for decision making. 

Indeed, two of the colleges that made limited progress at best in implementing culture 
of evidence practices had relatively strong IR staffs. The IR office in one of these colleges was 
readily able to carry out the longitudinal tracking and other analyses recommended by 
Achieving the Dream, but the college was not able to act on this information until a new 
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president arrived to replace the previous CEO, who had resigned and was reportedly not 
disposed to making decisions based on data. Another laggard college had perhaps the strongest 
IR department of any of the 13 colleges we studied. It was doing longitudinal tracking of 
student cohorts and other sophisticated analysis of student outcomes long before the college 
joined Achieving the Dream. However, the IR director was not part of the president’s cabinet, 
and neither the senior leadership nor the faculty and staff seemed to rely much on the 
information produced by the IR office in decision making. The challenge at this college was not 
getting data or doing research, but using the information produced by the IR office to improve 
programs and services. 

At the same time, some of the colleges did struggle with a lack of IR capacity. For 
example, one WA college was only able to make limited progress in building a culture of 
evidence because the IR director, like many staff members at this small, rural college, had to 
“wear many hats,” making it difficult to find time do the sorts of data analysis called for by 
Achieving the Dream. At least three other colleges struggled because of the turnover among, or 
limited capabilities, in their IR staffs. A mral PA college was unable to fill its IR staff position 
despite continuing efforts to do so. In lieu of an IR staff, the college relied on faculty and staff 
participation in “process management teams” to examine data on student outcomes and research 
promising practices for overcoming the achievement gaps identified. In addition, the college 
restmctured the job duties of an information technology (IT) analyst to that of a data mining 
specialist to do quantitative data analyses that would have been done by an IR staff person. 
Some of that person’s IT duties were redistributed to other staff. Thus, it was able to make 
promising progress toward building a culture of evidence despite limited IR capacity. 

All six WA colleges and two of the PA colleges struggled with antiquated information 
technology (IT) systems. Nevertheless, some of them were able to find ways to get the 
information they needed to identify gaps in student achievement and devise strategies for 
addressing them. 

Evidence-based program review and planning 

Leader colleges were more likely to have implemented evidence-based program review 
and strategic planning systems than were colleges that had not made much progress in 
implementing the Achieving the Dream institutional effectiveness model. Yet, having a 
strategic planning process in place was not sufficient to bring about changes in programs and 
services. The president of the WA college that made the least progress in implementing the 
Achieving the Dream model established a strategic planning process five years ago. It was not 
clear how much the process relied on evidence of student success, though the process did 
include setting measurable goals and objectives; still, it did not seem to have had much effect on 
efforts to improve student success. 
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In general, evidenced-based budgeting was not well developed at any of the colleges, 
although two of the PA colleges were moving in that direction. 

Professional development to support a culture of evidence 

Most of the colleges were just beginning to consider how they could design 
professional development activities to promote a culture of evidence on their campuses. A PA 
college located in a depressed part of the state had a leadership academy that it used to develop 
leaders from within. One recent project of rising administrators who were taking part in the 
academy was “Building a Culture of Evidence,” in which participants developed and 
implemented projects that exemplified effective data collection and evidence-based decision 
making. The results of these projects were later shared with the college and board of tmstees. 



Comparison with Round 1 Colleges 

In the baseline evaluation of the first-round Achieving the Dream colleges, CCRC and 
MDRC categorized the Round 1 colleges using a taxonomy somewhat similar to the one used in 
this study of the PA and WA colleges (Tables C.l and C.2). The authors of the report^^ on that 
earlier study classified the 27 Round 1 colleges as follows: 

• Six Round 1 colleges “were making clear progress toward institutionalizing a 
‘culture of evidence’” in that they were engaging faculty and staff on a 
substantial scale in using data and working together to improve student 
success, and had begun to use evidence of student outcomes as the basis for 
academic program evaluation, strategic planning, and budgeting. 

• Five had taken the important steps of engaging faculty and staff in the 
analysis of data on student outcomes and of adopting evidence-based 
strategic planning procedures. 

• Ten had some of the building blocks of a culture in evidence in place, 
including a well-developed institutional research capability and strategic 
planning process, but none had engaged a broad segment of faculty and staff 
in using data to improve programs and services; others were hampered by 
turnover of key project or college leadership. 

• Six colleges had limited data collection and analysis capabilities and had not 
begun using data to evaluate and improve programs and services. 



'^Brock et al. (2007), pp. 91-97. 
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Fewer than half of the first-round colleges were making progress toward 
institutionalizing a culture of evidence at a similar stage of the project. In contrast, 8 of 13 
Round 3 colleges were making solid progress toward institutionalizing a culture of evidence. 
All but two of the seven PA colleges were making good progress, although only half of the WA 
colleges were progressing apace. This difference in progress may have something to do with the 
fact that the PA colleges had to compete for the grants (and one, Allegheny, joined with its own 
funds). Based on this admittedly small sample, it seems as though the Round 3 colleges, and 
particularly the PA colleges, were making faster progress toward institutionalizing a culture of 
evidence. From our interviews with them, the PA and WA colleges (and presumably their 
coaches and data facilitators as well) seem to have benefited from the experiences of the earlier- 
round colleges. Still, as in the first round, there was a group of PA and WA colleges that were 
not making good progress. Whether they can catch up, and whether the colleges that were 
progressing can maintain their momentum, are questions to be answered in a second wave of 
research to be conducted by CCRC and MDRC in two years. 



Summary 

At the time of our visits in spring 2008 the research team rated 2 of the 13 PA and WA 
colleges as having made little or no progress toward institutionalizing a culture of evidence; 3 as 
having made limited progress, although major obstacles remained; 4 as having made promising 
progress; and 4 as having begun to institutionalize a culture of evidence on their campuses. The 
factors that distinguished colleges in the two top groups from those in the bottom groups 
included the following: leaders committed to making substantial changes in policy and 
investing the resources needed to support such changes, greater engagement of faculty and staff 
in the improvement process, strong collaboration between faculty and student services staff, and 
good communication across departments and divisions. 

Based on this limited sample, it seems that the PA and WA colleges, which joined the 
initiative in the third round, were, as a group, making faster progress toward institutionalizing a 
culture of evidence than the colleges that joined in the first round at a similar stage in the 
process. Based on our interviews, the PA and WA colleges seemed to have benefited from the 
experiences of the colleges that joined the initiative in the first two rounds. 
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Chapter 7 



The Impact of Achieving the Dream and 
Recommendations for Improvement 

This chapter examines how much Achieving the Dream contributed to progress made 
by the colleges to date in building a culture of evidence for student success. It also considers the 
extent to which the Achieving the Dream activities at the PA and WA colleges were focused on 
increasing equity of achievement among students by race and ethnicity and by income, as 
opposed to improving outcomes for all students. It further assesses how much Achieving the 
Dream coaching and other supports helped colleges progress toward the goal of closing the 
achievement gap. The chapter concludes with recommendations for additional actions that the 
Achieving the Dream initiative and the colleges themselves can take to increase the likelihood 
for fundamental improvement in outcomes. 



Initial Effects of Achieving the Dream 

Achieving the Dream has had positive effects on nearly all of the PA and WA colleges 
involved, including those that made little or no overall progress toward institutionalizing a 
culture of evidence. 

Effect on the Colleges Beginning to Institutionalize a Culture of Evidence 

The four colleges identified in Chapter 6 as those that made the greatest progress 
toward institutionalizing a culture of evidence had already taken steps toward the goals of 
Achieving the Dream before they became involved in the initiative. Even so. Achieving the 
Dream helped to accelerate the transformation at all of them. 

For example, the president of one of these colleges, located in PA, had made, as a 
centerpiece of her administration, a focus on student success and “management by fact,” 
spearheading a student success initiative even before the college joined Achieving the Dream. 
The college had an “institutional effectiveness model” with performance metrics that it used to 
measure progress toward the goals of its strategic plan. The college had a strong institutional 
research (IR) department that had collected and analyzed longitudinal data for some time as part 
of its institutional effectiveness model. Data on the performance metrics, as well as a host of 
information on student performance in the college’s “fact book,” were widely available to 
college personnel through the college’s intranet. The college used this information extensively 
in program review and strategic planning. 
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Even though this college was already moving in the direction of building a culture of 
evidence before it joined the initiative, Achieving the Dream provided a framework for 
analyzing data on students that had been lacking, according to individuals we interviewed at the 
college. The college used the Achieving the Dream performance measures to gauge its progress, 
drawing on data in the “monster database” that it established for tracking students. The college 
was also taking steps to make data on student progression and outcomes more accessible to 
faculty and staff. 

According to the president and other individuals we interviewed at the second PA 
college that was beginning to institutionalize a culture of evidence. Achieving the Dream had 
helped to increase discussions about student success across the campus. The dean of student 
affairs at the college said: “I truly think that one of the things that is happening is that we as an 
institution are talking about student success across the institution, and that in itself is a change.” 
As a result of Achieving the Dream, IR was more integrally involved in management decisions, 
rather than playing a background supporting role as in the past. The college was also using the 
Achieving the Dream goals and institutional improvement process as the framework for its 
strategic planning, budgeting, and reaccreditation efforts. 

Achieving the Dream was also providing a framework for student success efforts at a 
WA college with a culture of multicultural inclusion and commitment to success for all students 
that predated Achieving the Dream. The college’s leadership used the initiative as an 
opportunity to bring together and strengthen the many existing strands of work focused on 
improving student progress and outcomes and to do so through increased evidence-based 
decision making. According to college leaders, before it joined the initiative, the college had 
used data for decision making, but in a limited fashion. As a result of Achieving the Dream, use 
of data became much more systematic, and it was expected that decision makers would use 
evidence to inform their decisions about programs and services. According to the IR director at 
the college: 



There was no systematic plan for identifying achievement gaps. As problems 
or specific issues emerged, like students on probation, research would head 
in that direction, but it wasn’t a systematic review process. The 
implementation of our data team really got our analytical resources together 
to look at our data, disaggregate it, and to look at our achievement gaps and 
to decide on what interventions we wanted to adopt. The Achieving the 
Dream structure focused us on the gaps we want to address. 

Senior college administrators said that Achieving the Dream also led to a shift from 
anecdote to evidence in the college’s program review and strategic planning process. The vice 
president for student affairs said that participation in the initiative stimulated increased use of 
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data by her staff, who were using the Community College Survey of Student Engagement 
(CCSSE) and other information to see how student services could be strengthened. 

The other WA college that moved to institutionalize culture of evidence practices had a 
tradition of monthly campus-wide meetings where faculty and staff discussed student success 
issues. However, in the past IR had not played a prominent part in these discussions, and there 
was no systematic evaluation of program effectiveness to inform them. The vice president of 
instruction said that in the past institution-wide decision making had largely been made by “gut 
instinct” and there had not been a strong culture of using data and research. Achieving the 
Dream led the college to strengthen its IR office and refocused IR efforts from a primary 
emphasis on compliance reporting toward using research on students to inform improvements 
in programs and services. According to the new director of institutional research, previous 
evaluation efforts at the college were not systematic in nature, but rather tended to focus on 
specific programs. The IR staff is trying to help their colleagues throughout the college ask 
questions of the data that are more aligned with the college’s goals for improved student 
success. 



Effect on Colleges Making Promising Progress 

The four colleges that made promising progress toward building a culture of evidence 
(though they still faced obstacles) all began essentially from scratch when they joined the 
initiative. Indeed, three of them had had no IR staff. 

The president of one of these colleges indicated that she was initially skeptical about the 
need to establish an IR office, believing that college personnel knew what worked and what did 
not. Encouragement from the coach and data facilitator, combined with the initial experience of 
using data during the planning year, convinced the president of the need for an IR office. The 
college appointed a well-respected faculty leader to head a new office of institutional 
effectiveness (IE). The IE director made regular presentations to internal college constituencies 
about using data and designing interventions, and, according to faculty members we 
interviewed, they and their colleagues have begun to “embrace evidence-based decision 
making” and the campus is now “hungry” for data. 

A second college that started the initiative with no IR department probably came further 
than any other PA or WA college. It added a two-staff member IR office and combined IR with 
information technology (IT) to create a larger department focused on using data to improve 
programs and services. The newly formed and staffed IR office was responding to “a flood of 
data requests,” according to the director. The IR staff was developing a website to distribute 
information more quickly and cut down on requests from users. Concurrent with this increased 
investment in IR, the college took steps to involve faculty and staff in the improvement process. 
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For example, during the planning year, the college convened a group of faculty to examine the 
college’s cut score policies for its ACCUPLACER placement exam. Previously, the faculty had 
never been asked for input on placement policies. The experience helped developmental and 
college-level faculty understand the issues related to the use of the current placement test. They 
recognized that the mechanisms being used to place students into math courses did not make 
sense and that the sequence of math courses was not effective in helping students advance to 
college-level math. As a result, the faculty reorganized the math curriculum and was planning to 
evaluate the new configuration to see if student outcomes improved. Faculty we interviewed 
said that, through Achieving the Dream, they became aware that there was too much variation 
in the amount and quality of the education and services that students received, and that this 
variation was detrimental to student success. To achieve greater consistency across instructors 
and courses, the faculty had begun to work on common syllabi and expected learning outcomes 
for all courses. 

The third college that made promising progress toward building a culture of evidence 
still had no IR staff, despite continuing efforts to hire personnel. As described in Chapter 6, the 
college organized faculty and staff into process management teams to examine the effect of 
college policies on student success and to recommend changes to improve programs and 
services. 

Although the fourth promising progress college did have an IR office when it joined the 
initiative, the office moved beyond the compliance reporting function it primarily had in the 
past and assumed a much more prominent role in efforts to improve student success as a result 
of the college’s experience with Achieving the Dream. This was the first time that the college 
had done longitudinal tracking of students. The achievement gaps among students by race “got 
people’s attention,” according to the vice president for academic affairs, and led to the creation 
of a task force on developmental education, which generated a lot of enthusiasm among faculty 
and staff. Also, instead of analyzing grade distributions in individual courses only, as it did in 
the past, the college began looking at the progression of students from one course to the next. 
The vice president of student services said that doing so led to a more holistic view of student 
success in contrast with the past, when student success efforts were mostly “boutique” efforts 
focused on specific groups of students. According to this vice president, there was a growing 
awareness at the college that bringing about change on a meaningful scale requires a holistic, 
“systems” approach. 

Additional Effects for the PA and WA Colleges 

Participation in the initiative had other benefits for the PA and WA colleges, including 
for the five colleges that made at most limited progress toward building a culture of evidence. 
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• All but one or two PA colleges, and all but one WA college, saw Achieving 
the Dream as an “umbrella” for other student success initiatives. 

• Seven colleges added IR staff and another seven purchased data analysis 
software or upgraded their systems as a result of Achieving the Dream. 

• Achieving the Dream led six colleges to change their committee structure to 
allow for a greater focus on student success. 

• Respondents at four of the PA colleges and all of the WA colleges said that 
involvement in the initiative was helping their college prepare for or comply 
with accreditation requirements. 

• Respondents at four of the PA colleges and all of the WA colleges said that 
the initiative was helping them meet statewide performance accountability 
requirements. All six WA colleges mentioned that Achieving the Dream was 
providing a framework for college efforts to improve student outcomes under 
the state’s new Student Achievement Initiative, a new performance funding 
policy that rewards colleges for improving the rate at which students progress 
through college. 

Leadership turnover and somewhat strained relations between faculty and 
administrators caused one PA college to make only limited progress toward institutionalizing a 
culture of evidence. Achieving the Dream nevertheless helped the college’s research staff 
become much more proactive in efforts to improve student outcomes. The interim vice 
president for student development described the impact of the initiative: 

We have been lackadaisical about evaluating our programs. Achieving the 
Dream comes in and says “not only do you need to evaluate your learning 
and services, we also want you to be responsible for improving those 
outcomes.” Our research end was not focused on the learning environment. 

So [the initiative] came in and really centered us. Some of the changes we 
took on [as a result of the data analysis done under Achieving the Dream] 
were monumental, such as late registration. We have changed the policy so 
that a D in developmental education is no longer a passing grade. Now we 
are working on the first-year experience. It got us on the path of doing some 
things we needed to do for all our programs. We are much more data-focused 
than we were before. 
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Emphasis on Equity 

Two of the PA colleges and four of the WA colleges developed student success 
strategies designed expressly to address gaps in achievement by race/ethnicity or income. Most 
of them based these strategies on analyses of student outcomes data that indicated gaps in 
achievement among minority or low-income students. For example, an administrator at a WA 
college said her college’s strategies were based on data showing that low-income students and 
students of color were stmggling. She said that Achieving the Dream “shed light on practices in 
basic skills areas, basic skills courses, how they were affecting minorities and low SES students. 
It brought to light things that were not at the forefront. It was very shocking to many on 
campus.” 



Most of the colleges in both states did not attempt to make inequities in achievement a 
college-wide focus and priority, however. One exception was a WA college that came to 
Achieving the Dream with a strong culture of multicultural inclusion and a commitment to 
success for all students, including those who faced substantial barriers to success. Although this 
priority predated Achieving the Dream, the initiative was seen by many of the individuals we 
interviewed as a framework for connecting the many efforts on campus to improve outcomes 
for disadvantaged students. 

The president at a PA college tried to shed light on the problem of achievement gaps 
between white and minority students by, among other efforts, establishing a “diversity council.” 
However, the president was not satisfied that the college was doing enough to address the issue. 

A vice president of instmction and student services at a WA college described how 
Achieving the Dream allowed the college to have difficult conversations about the achievement 
gaps in student outcomes: 

[Achieving the Dream] provided a framework for having the hard 
conversations about race and ethnicity and underrepresented students. You 
need to have the conversation about what is needed to be done differently. 

You have to stress that it isn’t anyone’s fault.... So, [the initiative] provided 
the framework to have those conversations, to talk about minority and low- 
income students. The college wasn’t afraid of having those conversations as 
much as it didn’t know how to go about having them. 

Nevertheless, other personnel at this college indicated that the college still had far to go 
in opening the dialogue around the impact of race and ethnicity on student outcomes at the 
college. 



As previously discussed, at some colleges concern was expressed that targeting 
particular groups of students for special support was unfair to other students. For example, one 
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WA college was struggling to figure out how to meet the needs of the college’s growing Latino 
student population, a contentious issue at the college. The president and others wondered 
whether it was fair to expend a disproportionate amount of resources on a particular group of 
students. Further, there were only two Latino faculty members and they felt overworked in part 
because they were continually being asked to lead diversity efforts. They wished that the college 
would do more to increase diversity of the faculty and staff. However, increasing faculty and 
staff diversity was not a strategy being undertaken at this or any of the PA or WA colleges, at 
least as part of Achieving the Dream. 



The Value of the Achieving the Dream Supports 

Coaches and Data Facilitators 

Coaches and data facilitators were seen by most colleges as a particular strength of the 
Achieving the Dream initiative design. Many colleges saw their coach and data facilitator as a 
team (probably because they often visited together), and considered them to be mentors in the 
institutional change process. They were generally viewed as providing both a critical, outside 
perspective on the college’s progress, as well as serving as advocates on behalf of the college 
with the initiative. For example, the vice president of student services at a WA college said that 
the coach and data facilitator encouraged the college to dig deeper into their data on students: 

They provide a look from outside and as we ask questions they can say, “are 
you sure that’s what your data [are] telling you?” They’re good at getting us 
to ask those questions, [to] see things in data that you want to chase down the 
avenue. 

At another WA college, the coach and data facilitator were instrumental in “bringing 
around” the president who was initially reluctant to recognize the importance of institutional 
research and of hiring IR staff when the college had none before. 

The president of yet another WA college said of the coach and data facilitator: “They 
have been tremendous. Having coaches has been a great idea. This is critical. That is another 
reason that this initiative shines above most. I can call [our coach], president to president. She 
can say things to the college community that are difficult for me to say.” 

Coaches 

Because most of the coaches were former college presidents, college leaders said that 
they could turn to them for practical, useful advice. According to the presidents we interviewed, 
the coaches helped them engage various college constituencies in the institutional improvement 
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process. Colleges frequently called on their coaches to make presentations and lead discussions 
about the initiative and its goals with college stakeholders and to try to facilitate widespread 
buy-in among faculty and student services staff. At several colleges, coaches spoke at college 
fomms, including as a keynote speaker at the fall convocation of at least one college. College 
leaders looked to coaches to provide an outside voice to encourage faculty and staff to embrace 
the development of a culture of evidence. 

Most of the colleges in both states were satisfied with their coaches. Two PA colleges 
indicated that they did not consider their coach a good fit for their institutions. The president of 
another PA college said that he did not call on his coach much after the planning year: “I don’t 
know how much coaching we need. We were further along. Early in the process it was helpful; 
it’s less necessary now that we’ve gotten into implementation.” 

Data facilitators 

Similarly, all but one of the colleges found their data facilitator to be helpful. The 
exception was a case where a seasoned IR director at a college felt that the college’s facilitator 
did not have enough direct experience with institutional research to be very helpful to the 
college. 



Most of the other colleges were effusive about their data facilitators. For example, the 
president of a WA college said: “Our data coach came and spent three days with [our new IR 
director]. That was worth its weight in gold.” 

Multiple respondents at one WA college indicated that its data facilitator was able to 
serve as a helpful resource by providing examples of how other colleges approached similar 
challenges. The director of institutional effectiveness compared the data facilitator to a 
dissertation advisor: “A specialist in student success initiatives who reviews the college’s plans 
in order to make them stronger.” The director further described how the math department found 
the data facilitator very helpful when he met with them during the fall of the planning year to 
brainstorm about strategies could be piloted on a scale large enough to evaluate. 

At other colleges, the data facilitator helped lend creditability to the information coming 
from the college’s own institutional research staff. The IR staff at one WA college struggled 
from a lack of credibility with the college community because they were so new to the 
institution. The IR director said that the data facilitator was tmsted as an outside authority and 
so his support was critical. “Because of our newness, we’d recommend something and it was 
sometimes challenged. A little like ‘do you know what you’re doing? Check with [the data 
facilitator].’ And [the data facilitator] would back us up.” 



112 




The Achieving the Dream Database 

All but 2 PA colleges made some use of the national database in the initial analyses 
they conducted as part of the planning phase. The 2 that did not use it at all had very well 
developed in-house data systems and IR staff with experience doing longitudinal data analysis. 
Most of the other colleges relied more on their own data than on the national database. 
However, 5 of the 13 colleges found the Achieving the Dream database to be a useful stmcture 
for looking at own their data. 

A handful of the 13 PA and WA colleges planned to use the national dataset to analyze 
student outcomes as they moved beyond the initial analysis for Achieving the Dream. One of 
the WA colleges planned to use the e-STATS data analysis software the initiative has made 
available to colleges to compare itself to other colleges. However, a PA IR director had tried to 
use e-STATS to compare her college to others and could not because the college’s data 
presented in e-STATS seemed to have errors. This person tried unsuccessfully to get support 
from partner organizations. 

Strategy Institutes 

In general, interview respondents who attended any of the annual Achieving the Dream 
Strategy Institutes found them useful. Several respondents said that the opportunity to meet with 
colleagues from earlier-round colleges was particularly useful. A vice president of student 
services at a WA college said: 

I went to [the Strategy Institute in] Albuquerque.... Being in the strategy 
meeting with other colleges was extremely helpful — we were in meetings 
with people trying what we were trying and they helped us find land mines 
before we stepped on them. That was huge. They said no matter what we did 
it came back to “it’s not how we experience the student that’s important, it’s 
how they experience us.” When we realized that difference it changed 
everything we did. We looked at that and it helped us rethink where we 
wanted to go with our strategies. We aren’t fully there yet, but we’re headed 
in a very positive direction. 

Several respondents also said that they valued having time with colleagues from their 
own institutions. A PA IR director said: “Being [at the institute] provided an opportunity to 
spend time with colleagues. ... I was able to interact with coworkers differently than I do here.” 

Respondents at three colleges at least had more mixed reviews of the institutes. One PA 
president indicated that while he found the Kickoff Institute helpful, the subsequent Strategy 
Institute in Atlanta was not as useful. Another PA core team leader expressed fmstration that 
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presentations at the Strategy Institute attended by college team members were too focused at the 
classroom level, and didn’t provide enough guidance on how to bring about systemic reforms at 
the broader institution level. 



Suggestions for Improvement 

At every college we visited, we asked the individuals who were involved with 
Achieving the Dream if they had any suggestions for ways that the initiative or their own 
college performance could be improved. Some of their ideas are presented below. 

Increase Opportunities to Share with Other Colleges 

A very common suggestion was to increase opportunities to learn what other colleges 
are doing. One root of this recommendation was the colleges’ curiosity about how they were 
progressing in the initiative compared with other colleges. Many college leaders indicated that 
they relied on the coaches and data facilitators to give them feedback, and that the Strategy 
Institute sessions were also helpful in enabling them to see how advanced other colleges were in 
their work. Still, they would have liked to have had more information about how much progress 
other colleges were making, what strategies they were pursuing, and what was working and 
what was not. According to the president of a WA college: 

The piece that I’ve been disappointed with in Achieving the Dream is the 
ability to share information among similar institutions, [of having] a better 
sense of the work and performance of other institutions that are like ours.... 

We [in Achieving the Dream] haven’t found a way to work together 
nationally that has met my hopes yet. But it is a work in progress. 

Increase the Use of Personnel from Achieving the Dream Colleges 
as Coaches for New Colleges 

One college particularly benefited from having a data facilitator who was herself from a 
Round 1 college. The IR office, who found the facilitator especially helpful, said: “Because of 
her experience in a Round 1 college, she has been very helpful. She provides a lot of feedback 
to the college as a whole. She really knows her stuff.” This was the second data facilitator that 
the college was assigned: “Our first data facilitator did not have that experience, so she could 
only be so helpful.” The director, therefore, suggested that Achieving the Dream should take 
greater advantage of the cadre of community college personnel who were gaining experience in 
the initiative’s approach to coach and consult with colleges that were new to the initiative. 
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Improve the Availability of Comparative Performance Data 

Colleges also wanted to know how they were faring in terms of student outcomes. The 
president of a PA college said: “Fm waiting ... to see more national data. That’s not available to 
us. Td like to get a sense of us in the larger Achieving the Dream context.” As mentioned, some 
of the colleges that tried to do comparative analyses using e-STATS were disappointed because 
the data seemed to contain errors. 

Expand Opportunities and Support for Faculty Involvement 

Finding ways to involve faculty and staff in the process of using data to improve 
programs and services was perhaps the most common challenge facing the PA and WA 
colleges. The colleges that were further along in institutionalizing a culture of evidence had 
generally been more successful in engaging faculty in particular, but even for them, faculty 
engagement was still a work in process. Some respondents said they hoped that the initiative 
would provide clearer guidance to colleges on how effectively to engage faculty. Others 
indicated a need for more opportunities to involve faculty in discussions across campuses about 
Achieving the Dream goals and approaches. Some suggested that the Strategy Institutes were 
not an ideal forum for faculty engagement. For example, the vice president of student services at 
a PA college argued that pulling college personnel away from their jobs for four days during the 
early part of the spring term was very disruptive. Such scheduling made it especially difficult to 
involve faculty, who would have to miss nearly a week of teaching. The vice president 
suggested scheduling the Strategy Institute for early summer, after school ended. In addition, he 
recommended that Achieving the Dream sponsor “webinars” and shorter statewide or regional 
meetings for faculty and administrators during the school year. 

Rethink National Expansion Plan 

One PA president expressed concern about the proposal under consideration for 
Achieving the Dream to move to a fee-for-service model. He believed it would not attract the 
same level of participation, particularly for colleges that did not understand that they had an 
achievement gap. He further argued that there was a steep learning curve for colleges with the 
Achieving the Dream improvement approach. At the Atlanta Strategy Institute where this idea 
was first presented, he said that the consensus among presidents with whom he spoke was to 
continue to provide grant support for new colleges joining the initiative and then to decrease the 
amount of funding as the colleges gained experience with the process. 
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Summary 

While some of the PA and WA colleges made more progress than others in moving 
toward a culture of evidence, Achieving the Dream has had positive effects on nearly all of the 
colleges. For some, the initiative provided a framework for analyzing data on student 
progression and outcomes that helped to focus college personnel on gaps in student 
achievement and motivated them to find ways to address those gaps. 

Perhaps the most impressive effects were on the four colleges that had made “promising 
progress” (though obstacles remained) toward institutionalizing a culture of evidence. Three of 
these colleges had no IR staff when they began the initiative. At all four of these colleges. 
Achieving the Dream provided the impetus not only to strengthen IR capacity, but to give IR 
more of an integral role in decision making. Even among the five colleges that made little or no 
overall progress toward institutionalizing a culture of evidence, participating in Achieving the 
Dream had benefits, including helping them prepare for or comply with accreditation 
requirements, and providing an “umbrella” to help coordinate and focus other student success 
efforts at the college. 

Six of the PA and WA colleges developed student success strategies designed expressly 
to address gaps in achievement by race/ethnicity or income. Most of the colleges in both states, 
however, did not attempt to make remedying inequities in achievement a college-wide focus 
and priority. 

Most of the colleges viewed the coaches and data facilitators positively, appreciating 
the way they both provided a critical, outside perspective on the college’s progress and served 
as advocates on behalf of the colleges with the initiative. Most also made some use of the 
national database in the initial analyses they conducted as part of the planning phase, but most 
of them relied more on their own data than on the national database. In general, interview 
respondents who attended any of the annual Achieving the Dream Institutes found them useful. 

A number of individuals we interviewed at the PA and WA colleges suggested ways 
that the initiative could be improved. Some focused on ways for Achieving the Dream colleges 
to learn from each other’s experience. 

The PA and WA colleges clearly benefited from the experience of the colleges that 
joined the initiative in earlier rounds. As a group, the PA and WA colleges were further along in 
implementing the Achieving the Dream five-step improvement process than were the first- 
round colleges at a similar stage of their involvement. Our interviews at the PA and WA 
colleges suggest that they were able to accelerate their work on the process because of lessons 
learned from the earlier round colleges through conversations at the one of the Achieving the 
Dream Strategy Institutes and through information shared by their coaches and data facilitators. 
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As Achieving the Dream now enters a national expansion phase, the new colleges that join the 
initiative will have a great deal to learn from the experience and insights gained by the PA and 
WA colleges on how to transform their organizations and cultures to improve outcomes for all 
of their students. 
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Appendix A 

Tool for Measuring Development of the Achieving the 
Dream Model of Effective Institutions 




Achieving the Dream: Community Colleges Count 

Appendix A 

Tool for Measuring Development of the Achieving the Dream 
Model of Effective Institutions 

Use the scale provided to indicate the extent to which the college has implemented or 
developed the practices listed under each principle of the Achieving the Dream institutional 
effectiveness model. 



Model Principle 1 : Committed Leadership 



Extent of Implementation 

Indicator 1.1. Vision and values Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


1.1a 


President has developed a clear vision for student 
success with active involvement by institutional 
stakeholders. 


□ 


□ 


□ 


□ 


□ 


1.1b 


President and senior leadership emphasize the 
importance of improving student outcomes, not just 
increasing enrollments. 


□ 


□ 


□ 


□ 


□ 


1.1c 


President and other senior leaders have made an explicit 
policy commitment, communicated to faculty, staff, 
students and community, to achieve equity in student 
success across racial/ethnic and income groups. 


□ 


□ 


□ 


□ 


□ 
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Extent of Implementation 

Indicator 1.2. Commitment Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


1.2a 


President and senior ieaders demonstrate a wiilingness to 
support changes in organizationai structures and 
practices as needed to support evidence-based 
improvements in programs and services. 


□ 


□ 


□ 


□ 


□ 


1.2b 


President and senior ieaders demonstrate wiiiingness to 
support realiocation of resources as needed to support 
ew'dence-basec/ improvements in programs and services. 


□ 


□ 


□ 


□ 


□ 


1.2c 


Facuity ieaders activeiy support a broad-based agenda to 
improve student success. 


□ 


□ 


□ 


□ 


□ 


1.2d 


Board has made an expiicit commitment to improve 
student success. 


□ 


□ 


□ 


□ 


□ 


1.2e 


President reguiariy informs the board about outcomes of 
the coilege’s students and the effectiveness of efforts to 
improve student success. 


□ 


□ 


□ 


□ 


□ 
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Model Principle 2: Use of Evidence to Improve Programs and Services 



E xtent of Implementation 

Indicator 2. 1. Information technology (IT) capacity Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


2.1a 


IT systems allow for user-friendly retrieval and analysis of 
data on groups of students by administrators, faoulty, and 
staff. 


□ 


□ 


□ 


□ 


□ 


2.1b 


IT staff oapacity Is adequate to meet the demand for data 
and Institutional researoh. 


□ 


□ 


□ 


□ 


□ 


2.1c 


Policies and procedures are In place to ensure integrity of 
data collected. 


□ 


□ 


□ 


□ 


□ 
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E xtent of Implementation 

Indicator 2.2. Institutional research (IR) capacity Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


2.2a 


IR staff members are adequately trained in data analysis, 
especially in cohort tracking techniques. 


□ 


□ 


□ 


□ 


□ 


2.2b 


IR staff capacity is adequate to meet demand for data and 
research. 


□ 


□ 


□ 


□ 


□ 


2.2c 


IR staff members are seen as responsive to requests for 
information from administrators, faculty, and staff. 


□ 


□ 


□ 


□ 


□ 


2.2d 


IR staff members are skilled at clearly communicating 
research findings to key audiences. 


□ 


□ 


□ 


□ 


□ 


2.2e 


IR staff routinely works with faculty and staff to analyze 
data on student success. 


□ 


□ 


□ 


□ 


□ 


2.2f 


IR staff produces information useful for program 
evaluation, strategic planning, and budgeting. 


□ 


□ 


□ 


□ 


□ 


2.2g 


IR staff actively educates college personnel on how to use 
data and research to improve programs, services, and 
institutional management. 


□ 


□ 


□ 


□ 


□ 


2.2h 


IR staff has more than an administrative support role (i.e., 
not just compliance reporting); IR function is integral to the 
management of the institution. 


□ 


□ 


□ 


□ 


□ 
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Extent of Implementation 



Indicator 2.3. Process for identifying achievement gaps Little 

or A 







None 

1 


2 


3 


4 


lot 

5 


2.3a 


Institution reguiariy coilects, anaiyzes, and reports data on 
the Achieving the Dream performance indicators and other 
student outceme measures. 


□ 


□ 


□ 


□ 


□ 


2.3b 


Institution routineiy coilects, analyzes and reports 
longitudinal data on cohorts of students to chart student 
progress; college reports changes in performance rates for 
different cohorts over time. 


□ 


□ 


□ 


□ 


□ 


2.3c 


Institution routinely disaggregates student cehort data by 
age, race, gender, income and other factors to identify 
gaps in achievement among student groups. 


□ 


□ 


□ 


□ 


□ 


2.3d 


Institution regularly reperts changes in attainment rates for 
entering student cohorts in one year with the rates for 
cohorts beginning in subsequent years. 


□ 


□ 


□ 


□ 


□ 


2.3e 


Institution regularly conducts surveys and focus groups 
with students, faculty, and staff to identify weaknesses in 
programs and services and opportunities for improvement. 


□ 


□ 


□ 


□ 


□ 


Comments: 
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Indicator 2.4. Process for diagnosing gaps and implementation 

formulating solutions ® 







None 

1 


2 


g 

3 


4 


A lot 
5 


2.4a 


Institution routineiy coiiects and uses quantitative and 
quaiitative data te diagnose the causes of gaps in student 
achievement. 


□ 


□ 


□ 


□ 


□ 


2.4b 


Institutien has an inventory of current and past efforts to 
address student achievement gaps and documentation on 
the effectiveness of each. 


□ 


□ 


□ 


□ 


□ 


2.4c 


Institution has estabiished evidence-based process for 
formuiating strategies te address student achievement 
gaps. 


□ 


□ 


□ 


□ 


□ 


Indicator 2.5. Process for evaluating impact of solutions 


Little 

or 

None 

1 


Extent of Implementation 
Deve 
lopin 

g 

2 3 4 


A lot 
5 


2.5a 


Institution routineiy evaiuates the effectiveness of efforts to 
improve student success. 


□ 


□ 


□ 


□ 


□ 


2.5b 


The institution’s approach to evaiuation is methodoiogicaiiy 
sound. 


□ 


□ 


□ 


□ 


□ 


2.5c 


Institution uses the resuits of such evaiuations to further 
improve poiicies, pregrams, or services. 


□ 


□ 


□ 


□ 


□ 


Comments: 
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Model Principle 3: Broad Engagement 



Extent of Implementation 

Indicator 3. 1. Faculty Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


3.1a 


Faculty meets regularly to examine course and program 
outcomes and develop strategies for Improving student 
success. 


□ 


□ 


□ 


□ 


□ 


3.1b 


Faculty uses data and research to design and evaluate 
programs and teaching strategies. 


□ 


□ 


□ 


□ 


□ 


3.1c 


Faculty is receptive to evaluation of the effectiveness of 
their programs and teaching methods. 


□ 


□ 


□ 


□ 


□ 


3.1d 


Faculty is centrally involved in evaluating academic 
programs and teaching strategies. 


□ 


□ 


□ 


□ 


□ 


3.1e 


Faculty is actively involved on committees and other 
bodies concerned with student success. 


□ 


□ 


□ 


□ 


□ 


3.1f 


A critical mass of full-time faculty regularly participates 
in efforts to identify, diagnose and solve problems with 
student achievement. 


□ 


□ 


□ 


□ 


□ 


3.1g 


Part-time or adjunct faculty members are routinely 
informed of institutional efforts to improve student 
success and enceuraged to participate in such efforts. 


□ 


□ 


□ 


□ 


□ 
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E xtent of Implementation 

Indicator 3.2. Student services staff Little 







or 

None 

1 


2 


Devel 

oping 

3 


4 


A lot 
5 


3.2a 


Student services staff meets regularly to assess and 
develop strategies for improving the impact of their 
services on student success. 


□ 


□ 


□ 


□ 


□ 


3.2b 


Student services staff uses data and research to design 
and evaluate services and strategies. 


□ 


□ 


□ 


□ 


□ 


3.2c 


Student services staff is centrally involved in efforts to 
evaluate the effectiveness of student support services. 


□ 


□ 


□ 


□ 


□ 


3.2d 


Student services staff is well represented on 
committees and other bodies concerned with student 
success. 


□ 


□ 


□ 


□ 


□ 


Indicator 3.3. Collaboration 


Little 

or 

None 

1 


Extent of Implementation 

Devel 

oping 

2 3 4 


A lot 
5 


3.3a 


Faculty and student services staff regularly collaborate 
on efforts to improve student success. 


□ 


□ 


□ 


□ 


□ 


3.3b 


Institution promotes cross-program and divisional 
collaboration to improve student success. 


□ 


□ 


□ 


□ 


□ 
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Indicator 3.4. Students 



Extent of Implementation 



Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


3.4a 


Institution routinely seeks input from students on ways 
to improve student outcomes. 


□ 


□ 


□ 


□ 


□ 


3.4b 


Institution routinely invites active student participation in 
efforts to improve student outcomes. 


□ 


□ 


□ 


□ 


□ 


3.4c 


Students are represented on committees and other 
bodies concerned with student success. 


□ 


□ 


□ 


□ 


□ 


Comments: 



Extent of Implementation 

Indicator 3.5. External stakeholders Little 

or A 

None lot 



1 2 3 4 5 



Institution seeks input from external stakeholders (such 
as other educational institutions, human service 
agencies, community groups, and employers) to identify 
3.5a causes of achievement gaps and inform the | | | | | | | | | | 

development of strategies for improving student 
success. 



3.5b 



Institution shares data and collaborates with secondary 
schools, higher education institutions, workforce boards 
and, other outside entities for the purpose of improving 
student access and attainment. 



□ □ □ □ □ 



Institution is actively involved in strategic partnerships 
3.5c with outside stakeholders aimed at improving student | | | | | | | | | | 

success. 
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Model Principle 4: Systemic Institutionai Improvement 



E xtent of Implementation 

Indicator 4. 1. Institutional management Little 

or 







None 

1 


2 


3 


4 


A lot 
5 


4.1a 


Institution has established a strategic planning process 
that is breadly inclusive. 


□ 


□ 


□ 


□ 


□ 


4.1b 


Institution has established strategic planning precess 
that relies on data to set geals for student success and 
measure goal attainment. 


□ 


□ 


□ 


□ 


□ 


4.1c 


Institution regularly evaluates its academic programs to 
determine how well they promote student suecess and 
how they can be improved. 


□ 


□ 


□ 


□ 


□ 


4.1d 


Institution regularly evaluates all of its student services 
to determine how well they premote student success 
and how they can be improved. 


□ 


□ 


□ 


□ 


□ 


4.1e 


Institution uses data en program effectiveness to guide 
budget and resource allocation decisions. 


□ 


□ 


□ 


□ 


□ 


4.1f 


The institutien’s leadership creates a climate that 
supports correotive aotion needed to improve student 
outcomes. 


□ 


□ 


□ 


□ 


□ 


4.1g 


Institution has incentive system (for example, a system 
of professional development plans tied to institutional 
goals for student success) that encourages faculty and 
staff te work together to improve student outeomes and 
to use data to guide the process. 


□ 


□ 


□ 


□ 


□ 


4.1h 


Institution uses external grant funds strategically to 
support systemio efforts to improve eutcomes for all 
students, not just for isolated projects that benefit small 
numbers of students. 


□ 


□ 


□ 


□ 


□ 


4.1i 


Institution actively seeks te scale up and sustain pilot 
programs or practices that preve effective. 


□ 


□ 


□ 


□ 


□ 
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Indicator 4.2. Organization 


Little 


Extent of Implementation 




or 

None 


A lot 




1 


2 3 4 5 



4.2a 


Administrative structure and staffing promotes cross- 
divisionai focus and action on improving student 
outcomes. 


□ 


□ 


□ 


□ 


□ 


4.2b 


Committee structure promotes cross-divisionai focus 
and action on improving student outcomes. 


□ 


□ 


□ 


□ 


□ 


4.2c 


Committees concerned with student success inciude 
representatives from key stakehoiders, such as facuity, 
student services staff, administrators and students. 


□ 


□ 


□ 


□ 


□ 


4.2d 


Committees concerned with student success reiy on 
data for decision making. 


□ 


□ 


□ 


□ 


□ 


Indicator 4.3. Hiring and professional development 


Little 

or 

None 

1 


E xtent of Implementation 
2 3 4 


A lot 
5 


4.3a 


institution considers commitment to student success as 
a key criterion in aii hiring decisions. 


□ 


□ 


□ 


□ 


□ 


4.3b 


institution encourages and supports professional 
development for faculty and staff to help them become 
more effective in facilitating student success. 


□ 


□ 


□ 


□ 


□ 


4.3c 


Faculty and staff on a wide scale participate in 
seminars, workshops, and conferences related to 
improving student success. 


□ 


□ 


□ 


□ 


□ 


4.3d 


Institution provides training to faculty and staff on using 
data and research to improve programs and services. 


□ 


□ 


□ 


□ 


□ 
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Appendix B 

Mean Institutional Rates for Achieving the Dream 
Performance Measures 




Achieving the Dream: Community Colleges Count 



Appendix Table B,1 



Average Institutional Rates on Achieving the Dream Performance Indicators at Pennsylvania Colleges (Round 3) 

Fall 2004 Cohort, Three- Year Outcomes 





Mean 
Value (%) 


Standard 
Deviation (%)“ 


Minimum 
Value (%)*" 


Maximum 
Value (%)® 


Number of 
Institutions 
Reporting 


Developmental courses"* 

Successful completion of highest-level developmental math course 


36.6 


14.5 


7.1 


51.3 


7 


Successful completion of highest-level developmental English course 


44.7 


16.4 


12.0 


61.3 


7 


Successful completion of highest-level developmental reading course 


34.0 


20.5 


5.0 


63.8 


7 


Gatekeeper courses 

Successful completion of gatekeeper math course® 


27.8 


12.3 


1.0 


50.7 


7 


Percent referred who enroll in gatekeeper math^ 


27.4 


14.9 


1.0 


50.7 


7 


Percent referred who complete gatekeeper math^ 


21.7 


11.9 


2.8 


42.4 


7 


Successful completion of gatekeeper English course® 


46.1 


5.3 


38.6 


54.5 


7 


Percent referred who enroll in gatekeeper English*" 


45.1 


13.8 


27.2 


64.6 


7 


Percent referred who complete gatekeeper English*" 


41.1 


11.4 


21.4 


57.6 


7 


Course completion 

Ratio of completed credits to attempted credits 


76.4 


13.3 


58.9 


93.7 


7 
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Appendix Table B.l (continued) 








Number of 


Mean Standard 


Minimum 


Maximum 


Institutions 


Value (%) Deviation (%)" 


Value {%f 


Value {%f 


Reporting 



Persistence 

Enrolled in the first semester after the initial term of enrollment or completed 
within first year' 

Enrolled in at least one semester in the second year or completed within two 
years-' 

Enrolled in at least one semester in each of the first three years or completed 
within 3 years'^ 

Completions 

Completed within 3 years 

Obtained an associate degree within 3 years 

Obtained a certificate or diploma within 3 years 

Enrolled in at least one semester in the third year 



70.1 


4.6 


62.0 


113 


7 


52.8 


4.0 


47.9 


58.6 


7 


30.3 


2.7 


26.5 


33.5 


7 



10.4 


5.1 


4.5 


19.9 


7 


9.1 


5.0 


4.4 


19.0 


7 


1.5 


1.0 


0.1 


2.5 


7 


31.2 


1.8 


27.4 


32.6 


7 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania Achieving the Dream colleges, 
which includes 21,501 students at seven community colleges. Eigures represent average institutional rates. 

Delaware County Community College notes: 

Delaware has a policy in which students who have test scores that are “below minimum entry” do not qualify for the lowest level of 
remediation; there is no course designed for these students. 

Most ESL students take a separate ESL exam to place into ESL classes. 
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Appendix Table B.l (continued) 



Montgomery County Community College notes: 

Anyone who attempts and/or completes any developmental courses does not receive college credit. 

System "wipes out" credits attempted if a student withdrawals from a college-level course. 

“The standard deviation is a calculated variable measure of the dispersion of values around the mean. 

’’The minimum value is the lowest rate calculated among institutions reporting data. 

“The maximum value is the highest rate calculated among institutions reporting data. 

‘’Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

’“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

®The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a 
course successfully. 

’“‘Percent referred” is the percentage of students who were referred to developmental instruction in English. 

'The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

^For the fall 2004 cohort, the second year is academic year 2005-2006. 

’‘For the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,2 



Average Institutional Rates on Achieving the Dream Performance Indicators at Washington Colleges (Round 3) 

Fall 2004 Cohort, Three- Year Outcomes 





Mean 
Value (%) 


Standard 
Deviation (%)" 


Minimum 
Value {%f 


Maximum 
Value {%f 


Number of 
Institutions 
Reporting 


Develonmental courses'* 

Successful completion of highest-level developmental math course 


26.9 


3.9 


21.6 


31.4 


5 


Successful completion of highest-level developmental English course 


41.0 


7.6 


32.8 


49.0 


5 


Successful completion of highest-level developmental reading course 


19.8 


19.5 


0.4 


43.8 


5 


Gatekeener courses 

Successful completion of gatekeeper math course‘s 


27.2 


9.6 


18.7 


43.1 


6 


Percent referred who enroll in gatekeeper math^ 


28.3 


11.1 


20.0 


48.8 


6 


Percent referred who complete gatekeeper math* 


25.4 


10.3 


18.4 


44.8 


6 


Successful completion of gatekeeper English course® 


41.5 


14.4 


17.3 


58.1 


6 


Percent referred who enroll in gatekeeper English*' 


38.5 


14.9 


10.5 


49.8 


6 


Percent referred who complete gatekeeper English*" 


34.1 


13.2 


9.2 


44.8 


6 


Course comuletiou 

Ratio of completed credits to attempted credits 


78.9 


4.5 


75.0 


87.9 


6 
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Appendix Table B.2 (continued) 












Mean 
Value (%) 


Standard 
Deviation (%)" 


Minimum 
Value (%)*’ 


Maximum 
Value {%f 


Number of 
Institutions 
Reporting 


Persistence 

Enrolled in the first semester after the initial term of enrollment or completed 


73.5 


20.5 


32.2 


88.6 


6 


within first year' 

Enrolled in at least one semester in the second year or completed within two years^ 


57.6 


6.9 


48.8 


68.6 


6 


Enrolled in at least one semester in each of the first three years or completed 


40.2 


9.9 


30.8 


58.6 


6 


within 3 years*' 




Comnletions 

Completed within 3 years 


27.2 


15.4 


13.5 


56.5 


6 


Obtained an associate degree within 3 years 


15.9 


5.2 


9.0 


21.9 


6 


Obtained a certificate or diploma within 3 years 


11.0 


19.6 


1.4 


50.9 


6 


Enrolled in at least one semester in the third year 


21.0 


9.0 


4.1 


28.7 


6 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Washington Achieving the Dream colleges, 
which includes 4,086 students at six community colleges. Eigures represent average institutional rates. 

Renton Technical College notes: 

Since Renton is a technical college, they have a problem coding credits (i.e. distinguishing attempted from earned). Eor the present, they 
decided to use their “credits enrolled” field as “credits attempted.” 

Renton does not have the ability to match scores to “college level” because they are a technical college. 

Remedial classes are taught in courses offered by Basic Studies department, but not as distinct remedial courses. 

Big Bend Community College does not have a referral system. They have assumed the student is referred to the level they placed. 
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Appendix Table B.2 (continued) 



Seattle Community College - Central Campus does not offer a developmental reading course. Developmental English is strictly a writing 
program. 

Yakima Valley Community College notes: 

Yakima has a placement policy that dictates the student take the class they place into; therefore, their placement is the referral. 

For students that were awarded more than one award, they selected the highest precedent first and then if there were duplicates, selected the 
last recorded award with associated CipCode. 

“The standard deviation is a calculated variable measure of the dispersion of values around the mean. 

’’The minimum value is the lowest rate calculated among institutions reporting data. 

“The maximum value is the highest rate calculated among institutions reporting data. 

‘’Renton Technical College does not have a remediation system below Level 1 (one level below college). As such, only five community colleges 
are evaluated in the “Developmental Courses” section. Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

’“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

®The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a 
course successfully. 

’“‘Percent referred” is the percentage of students who were referred to developmental instruction in English. 

'The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

^For the fall 2004 cohort, the second year is academic year 2005-2006. 

‘‘For the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,3 

Average Institutional Rates on Achieving the Dream Performance Indicators at 
Pennsylvania, Washington, and Round 1 Colleges, 

Fall 2004 Cohort, Three- Year Outcomes 





























Number of 












Standard Deviation 


Minimum Value 


Maximum Value 


Institutions 






Mean Value (%) 




(%r 






(%)'^ 






(%r 






Reporting 






PA 


WA 


Rd 1 


PA 


WA 


Rd 1 


PA 


WA 


Rd 1 


PA 


WA 


Rd 1 


PA 


WA 


Rd 1 


Develonmental courses"' 

Successful completion of 
































highest-level developmental 
math course 


36.6 


26.9 


28.9 


14.5 


3.9 


12 


7.1 


21.6 


5.7 


51.3 


31.4 


48.9 


7 


5 


22 


Successful completion of 
































highest-level developmental 
English course 


44.7 


41.0 


35.7 


16.4 


7.6 


16.5 


12.0 


32.8 


5.1 


61.3 


49.0 


68.0 


7 


5 


22 


Successful completion of 
































highest-level developmental 
reading course 


34 


19.8 


37.0 


20.5 


19.5 


17.5 


5.0 


0.4 


4.7 


63.8 


43.8 


66.6 


7 


5 


23 


Gatekeeper courses 

Successful completion of 
gatekeeper math course"^ 


27.8 


27.2 


20.5 


12.3 


9.6 


7.9 


1.0 


18.7 


6.8 


50.7 


43.1 


32.9 


7 


6 


22 


Percent referred who enroll in 
gatekeeper math^ 


27.4 


28.3 




14.9 


11.1 




1.0 


20.0 




50.7 


48.8 




7 


6 




Percent referred who 
complete gatekeeper math^ 


21.7 


25.4 




11.9 


10.3 




2.8 


18.4 




42.4 


44.8 




7 


6 




Successful completion of 
gatekeeper English course® 


46.1 


41.5 


30.1 


5.3 


14.4 


8.6 


38.6 


17.3 


15.6 


54.5 


58.1 


46.0 


7 


6 


23 
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Appendix Table B.3 (continued) 


Mean Value (%) 


Standard Deviation 

(%r 


Minimum Value 

(%f 


Maximum Value 

(%r 


Number of 
Institutions 
Reporting 


PA WA Rd 1 


PA WA Rd 1 


PA WA Rd 1 


PA WA Rd 1 


PA WA Rd 1 



Gatekeeper courses (continued) 



Percent referred who enroll in 
gatekeeper English*' 

Percent referred who 


45.1 


38.5 




13.8 


14.9 




27.2 


10.5 




64.6 


49.8 




7 


6 




complete gatekeeper 
English*' 


41.1 


34.1 




11.4 


13.2 




21.4 


9.2 




57.6 


44.8 




7 


6 




Course completion 

Ratio of completed credits to 
attempted credits 

Persistence 

Enrolled in the first semester 


76.4 


78.9 


70.1 


13.3 


4.5 


9.5 


58.9 


75.0 


51.9 


93.7 


87.9 


92.3 


7 


6 


29 


after the initial term of 
enrollment or completed 
within first year' 


70.1 


73.5 


72.4 


4.6 


20.5 


5.4 


62.0 


32.2 


56.3 


77.3 


88.6 


81.3 


7 


6 


29 


Enrolled in at least 1 semester 
































in the second year or 
completed within 2 years^ 


52.8 


57.6 


54.4 


4.0 


6.9 


6.4 


47.9 


48.8 


40.6 


58.6 


68.6 


66.6 


7 


6 


29 


Enrolled in at least 1 semester 
































in each of the E* 3 years or 
completed within 3 years’^ 


30.3 


40.2 


33.3 


2.7 


9.9 


7.6 


26.5 


30.8 


17.2 


33.5 


58.6 


46.1 


7 


6 


29 


Completions 

Completed within 3 years 


10.4 


27.2 


10.8 


5.1 


15.4 


6.8 


4.5 


13.5 


1.6 


19.9 


56.5 


27.6 


7 


6 


28 


Obtained an associate degree 
within 3 years 


9.1 


15.9 


7.3 


5.0 


5.2 


4.6 


4.4 


9.0 


0.9 


19.0 


21.9 


19.1 


7 


6 


28 


Obtained a certificate or 
diploma within 3 years 


1.5 


11.0 


3.5 


1.0 


19.6 


3.6 


0.1 


1.4 


0.4 


2.5 


50.9 


16.3 


7 


6 


28 


Enrolled in at least one 
semester in the third year 


31.2 


21.0 


28.7 


1.8 


9.0 


6.4 


27.4 


4.1 


14.3 


32.6 


28.7 


43.8 


7 


6 


28 
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Appendix Table B.3 (continued) 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 25,587 students at 13 community colleges, and sample members in the fall 2002 cohort at Round 1 Achieving the 
Dream colleges, which includes 66,129 students at 29 colleges. Eigures represent average institutional rates. Some colleges did not report into the 
database on some measures. 



“The standard deviation is a calculated variable measure of the dispersion of values around the mean. 

’’The minimum value is the lowest rate calculated among institutions reporting data. 

“The maximum value is the highest rate calculated among institutions reporting data. 

‘’Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

’“Percent referred” is the percentage of students who were referred to developmental instruction in math. These measures were not included in the 
baseline data for Round 1 Achieving the Dream colleges. 

®The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

’“‘Percent referred” is the percentage of students who were referred to developmental instruction in English. These measures were not included in the 
baseline data for Round 1 Achieving the Dream colleges. 

'The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

^Eor the fall 2004 cohort, the second year is academic year 2005-2006. 

*‘Eor the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,4 



Average Institutional Rates on Achieving the Dream Performance Indicators 
at Pennsylvania and Washington Colleges (Round 3), 
by Race/Ethnicity,® Fall 2004 Cohort, Three-Year Outcomes 





White, Non- 
Hispanic 


Asian, Pacific 
Islander 


Black, Non- 
Hispanic 


Hispanic 


Native American 




PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


Developmental courses'’ 






















Successful completion of highest-level 
developmental math course 


39.2 


27.7 


43.1 


28.8 


29.7 


23.1 


29.6 


22.2 


31.6 


27.4 


Successful completion of highest-level 
developmental English course 


47.6 


36.7 


42.2 


40.2 


41.9 


37.0 


37.3 


50.2 


36.6 


34.7 


Successful completion of highest-level 
developmental reading course 

Gatekeeper courses 


33.2 


20.6 


32.2 


15.3 


33.2 


20.9 


28.4 


25.4 


14.1 


42.9 


Successful completion of gatekeeper math 
course'’ 


29.5 


27.7 


29.1 


31.7 


18.9 


21.5 


24.0 


22.2 


24.8 


26.4 


Percent referred who enroll in gatekeeper math'* 


28.9 


29.6 


28.0 


28.7 


25.0 


24.7 


25.5 


20.2 


31.3 


22.7 


Percent referred who complete gatekeeper 
math'* 


22.9 


26.3 


22.8 


26.8 


18.3 


22.9 


20.7 


17.4 


31.3 


20.4 


Successful completion of gatekeeper English 
course*’ 


49.1 


35.8 


40.2 


46.9 


39.0 


44.4 


37.0 


36.3 


39.2 


39.7 


Percent referred who enroll in gatekeeper 
English^ 


47.7 


37.5 


43.9 


38.3 


40.7 


27.6 


39.2 


47.6 


32.5 


41.7 


Percent referred who complete gatekeeper 
English^ 


44.5 


34.1 


37.6 


32.3 


35.1 


25.5 


33.0 


37.0 


25.9 


29.9 
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Appendix Table B.4 (continued) 


White, Non- 


Asian, Pacific 


Black, Non- 






Hispanic 


Islander 


Hispanic 


Hispanic 


Native American 


PA WA 


PA WA 


PA WA 


PA WA 


PA WA 



Course completion 



Ratio of completed credits to attempted credits 

Persistence 

Enrolled in the first semester after the initial 


78.8 


80.3 


75.9 


83.7 


67.9 


69.3 


74.6 


73.1 


67.1 


75.7 


term of enrollment or completed within first 
year*^ 


71.2 


82.3 


68.0 


86.3 


64.9 


78.2 


67.1 


75.5 


55.3 


72.0 


Enrolled in at least one semester in the second 
year or completed within two years’" 


54.1 


58.3 


53.8 


63.3 


45.2 


46.5 


48.3 


50.1 


44.7 


58.5 


Enrolled in at least one semester in each of the 
first three years or completed within 3 years" 

Comnletions 


31.7 


40.7 


33.6 


47.4 


23.2 


30.5 


26.2 


32.5 


28.5 


37.9 


Completed within 3 years 


11.7 


29.2 


8.7 


31.1 


5.1 


14.9 


00 

00 


22.4 


1.2 


24.1 


Obtained an associate degree within 3 years 


10.3 


17.5 


6.6 


19.0 


4.1 


7.8 


7.6 


12.6 


1.2 


13.8 


Obtained a certificate or diploma within 3 years 


1.6 


11.5 


2.2 


13.5 


0.9 


7.5 


1.2 


9.1 


0.0 


10.3 


Enrolled in at least one semester in the third 


31.9 


19.7 


35.6 


21.6 


26.5 


18.2 


28.8 


20.3 


38.1 


18.5 



year 

SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 25,587 students at 13 community colleges. Eigures represent average institutional rates. Cases where a particular 
racial/ethnic group for the institution’s cohort sample had observations of five or less were censored. 



142 








Appendix Table B.4 (continued) 



“The racial/ethnic category “Other” was excluded from the analysis. This group includes those 1,227 students identified as “Multiracial,” “Nonresident 
alien,” “Other,” or “Unknown.” 

’’Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

“"“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

“The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

^“Percent referred” is the percentage of students who were referred to developmental instruction in English. 

®The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

’’Eor the fall 2004 cohort, the second year is academic year 2005-2006. 

'Eor the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,5 



Average Institutional Rates on Achieving the Dream Performance Indicators at Pennsylvania and Washington Colleges 
(Round 3), by Race/Ethnicity** Among Female Students, Fall 2004 Cohort, Three- Year Outcomes 





White, Non- 
Hispanic 
Eemales 
PA WA 


Asian, Pacific 
Islander Eemales 
PA WA 


Black, Non- 
Hispanic 
Eemales 
PA WA 


Hispanic Eemales 
PA WA 


Native American 
Eemales 
PA WA 


Developmental courses’’ 






















Successful completion of highest-level 
developmental math course 


40.7 


29.3 


42.3 


33.7 


32.2 


26.2 


29.6 


20.4 


45.0 


22.9 


Successful completion of highest-level 
developmental English course 


50.1 


36.9 


40.6 


54.0 


48.7 


23.8 


42.3 


42.7 


47.2 


16.7 


Successful completion of highest-level 
developmental reading course 

Gatekeeper courses 


35.2 


22.1 


33.5 


14.3 


34.3 


22.8 


32.2 


22.5 


22.2 


50.0 


Successful completion of gatekeeper math 
course'’ 


28.6 


28.1 


27.2 


32.1 


17.0 


24.0 


25.5 


16.4 


19.7 


15.6 


Percent referred who enroll in gatekeeper 
math'* 


28.6 


29.3 


26.6 


35.4 


21.7 


28.6 


25.2 


18.5 


40.0 


22.9 


Percent referred who complete gatekeeper 
math'* 


22.8 


26.7 


23.6 


33.5 


14.8 


25.5 


20.7 


16.8 


40.0 


19.8 


Successful completion of gatekeeper English 
course*’ 


51.4 


46.0 


37.5 


49.0 


42.5 


47.7 


43.2 


36.6 


24.8 


45.2 


Percent referred who enroll in gatekeeper 
English^ 


49.7 


36.6 


40.9 


51.0 


43.4 


23.8 


41.9 


35.8 


42.2 


33.3 


Percent referred who complete gatekeeper 
English^ 


47.1 


33.5 


36.1 


41.1 


37.8 


19.6 


39.2 


31.7 


31.7 


33.3 
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Appendix Table B.5 (continued) 

White, Non- Black, Non- 

Hispanic Asian, Pacific Hispanic Native American 

Females Islander Females Females Hispanic Females Females 

PA WA PA WA PA WA PA WA PA WA 



Course completion 

Ratio of completed credits to attempted 
credits 

Persistence 

Enrolled in the first semester after the initial 
term of enrollment or completed within first 
year*^ 

Enrolled in at least one semester in the second 
year or completed within two years’" 

Enrolled in at least one semester in each of the 
first three years or completed within 3 years" 

Completions 

Completed within 3 years 

Obtained an associate degree within 3 years 

Obtained a certificate or diploma within 3 
years 

Enrolled in at least one semester in the third 
year 



80.5 


81.5 


78.7 


87.1 


69.1 


73.4 


83.0 


67.6 


82.2 


63.9 


55.2 


58.7 


55.6 


61.8 


48.3 


34.5 


43.1 


35.4 


48.3 


27.6 


13.2 


31.9 


10.2 


35.4 


6.2 


11.5 


17.4 


7.5 


19.8 


5.2 


1.9 


14.2 


2.6 


16.8 


1.0 


33.8 


19.4 


35.6 


20.8 


30.5 



72.2 


72.7 


70.3 


54.9 


81.2 


80.8 


68.6 


72.2 


54.9 


65.9 


52.3 


47.4 


44.9 


41.5 


55.5 


34.0 


28.4 


27.8 


18.1 


36.8 


15.8 


10.1 


16.2 


1.6 


22.2 


7.1 


00 

00 


9.7 


1.6 


18.6 


00 

00 


1.4 


4.0 


0.0 


3.6 


21.6 


30.0 


21.4 


28.0 


22.3 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 14,320 students at 13 community colleges. Eigures represent average institutional rates. Cases where a particular 
racial/ethnic group for the institution’s cohort sample had observations of five or less were censored. 



145 







Appendix Table B.5 (continued) 



“The racial/ethnic category “Other” was excluded from the analysis. This group includes those 1,227 students identified as “Multiracial,” “Nonresident 
alien,” “Other,” or “Unknown.” 

’’Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

“"“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

“The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

^“Percent referred” is the percentage of students who were referred to developmental instruction in English. 

®The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

’’Eor the fall 2004 cohort, the second year is academic year 2005-2006. 

'Eor the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,6 



Average Institutional Rates on Achieving the Dream Performance Indicators at Pennsylvania and Washington Colleges 
(Round 3), by Race/Ethnicity“ Among Male Students, Fall 2004 Cohort, Three- Year Outcomes 





White, Non- 


Asian, Pacific 


Black, Non- 






Native American 




Hispanic Males 


Islander Males 


Hispanic Males 


Hispanic Males 


Males 






PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


Develonmental courses'’ 






















Successful completion of highest-level developmental 
math course 


36.9 


25.7 


45.7 


34.0 


26.8 


24.2 


29.7 


23.5 


50.0 


14.3 


Successful completion of highest-level developmental 
English course 


44.9 


37.1 


44.7 


54.6 


43.6 


44.6 


31.2 


46.2 


60.0 




Successful completion of highest-level developmental 
reading course 


30.8 


14.2 


31.5 


16.8 


33.4 


17.8 


13.7 


16.7 


25.0 




Gatekeener courses 

Successful completion of gatekeeper math course'’ 


30.5 


27.3 


31.4 


30.2 


21.6 


21.3 


20.0 


20.2 


32.4 


0.0 


Percent referred who enroll in gatekeeper math'' 


29.2 


30.0 


31.5 


36.7 


27.7 


20.8 


17.2 


21.7 


50.0 


0.0 


Percent referred who complete gatekeeper math'' 


22.9 


26.0 


23.1 


33.8 


21.3 


19.1 


9.5 


17.8 


50.0 


0.0 


Successful completion of gatekeeper English course'’ 


46.5 


50.5 


44.4 


45.1 


34.1 


39.3 


40.2 


35.4 


43.6 


12.5 


Percent referred who enroll in gatekeeper English^ 


45.8 


38.1 


48.3 


45.7 


36.4 


41.3 


34.8 


37.5 


50.0 




Percent referred who complete gatekeeper English^ 


41.8 


36.1 


42.2 


42.5 


28.9 


40.1 


22.1 


28.4 


50.0 




Course comuletiou 

Ratio of completed credits to attempted credits 


76.5 


78.6 


72.2 


80.7 


66.1 


66.5 


64.9 


70.3 


75.9 


60.4 
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Appendix Table B.6 (continued) 




White, Non- 


Asian, Pacific 


Black, Non- 






Native American 




Hispanic Males 


Islander Males 


Hispanic Males 


Hispanic Males 


Males 






PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


PA 


WA 


Persistence 

Enrolled in the first semester after the initial term of 
enrollment or completed within first year® 


68.7 


81.6 


68.7 


89.0 


66.2 


80.1 


62.2 


81.2 


59.7 


70.8 


Enrolled in at least one semester in the second year or 
completed within two years'" 


52.8 


57.8 


51.3 


64.8 


40.6 


45.5 


45.4 


55.4 


47.8 


37.5 


Enrolled in at least one semester in each of the first 
three years or completed within 3 years' 


28.4 


37.9 


31.8 


46.8 


16.7 


26.1 


21.9 


29.5 


37.8 


31.3 


Completions 

Completed within 3 years 


9.9 


26.0 


6.9 


30.2 


3.5 


12.9 


4.0 


14.9 


1.7 


4.2 


Obtained an associate degree within 3 years 


8.9 


17.4 


5.4 


19.3 


2.6 


4.7 


2.0 


12.5 


16.7 


0.0 


Obtained a certificate or diploma within 3 years 


1.1 


8.5 


1.5 


11.0 


0.8 


9.4 


2.1 


1.5 


0.0 


4.2 


Enrolled in at least one semester in the third year 


29.7 


20.1 


36.0 


21.7 


20.7 


16.0 


27.7 


27.8 


48.6 


31.3 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 1 1,094 students at 13 community colleges. Eigures represent average institutional rates. Cases where a particular 
racial/ethnic group for the institution’s cohort sample had observations of five or less were censored. 
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Appendix Table B.6 (continued) 



“"The racial/ethnic category “Other” was excluded from the analysis. This group includes those 1,227 students identified as “Multiracial,” “Nonresident 
alien,” “Other,” or “Unknown.” 

’’Grades of C or better must be earned to have completed a course successfully. 

‘^The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

^“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

‘’The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

^“Percent referred” is the percentage of students who were referred to developmental instruction in English. 

®The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

’’For the fall 2004 cohort, the second year is academic year 2005-2006. 

‘For the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,7 

Average Institutional Rates on Achieving the Dream Performance Indicators 
at Pennsylvania and Washington Colleges (Round 3), by Pell Grant Receipt Status, “ Fall 2004 Cohort, Three-Year Outcomes 





PA Pell Recipient 


PA Pell 
Nonrecipient 


WA Pell 
Recipient 


WA Pell 
Nonrecipient 


Developmental courses'’ 










Successful completion of highest-level developmental math course 


44.3 


33.8 


31.9 


24.7 


Successful completion of highest-level developmental English course 


54.5 


40.8 


47.3 


36.9 


Successful completion of highest-level developmental reading course 


42.0 


30.0 


21.3 


18.8 


Gatekeener courses 










Successful completion of gatekeeper math course'’ 


32.7 


26.0 


28.8 


26.4 


Percent referred who enroll in gatekeeper math'* 


33.1 


25.1 


31.0 


26.9 


Percent referred who complete gatekeeper math'' 


25.6 


20.4 


28.8 


23.6 


Successful completion of gatekeeper English course'’ 


58.7 


41.4 


51.5 


44.1 


Percent referred who enroll in gatekeeper English' 


54.6 


42.0 


43.9 


35.0 


Percent referred who complete gatekeeper English' 


49.6 


38.4 


39.0 


30.8 


Course comuletion 










Ratio of completed credits to attempted credits 


75.0 


76.8 


78.7 


79.3 
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Appendix Table B.7 (continued) 






PA Pell 


WA Pell 


WA Pell 




PA Pell Recipient 


Nonrecipient 


Recipient 


Nonrecipient 


Persistence 










Enrolled in the first semester after the initial term of enrollment or completed 
within first year® 

Enrolled in at least one semester in the second year or completed within two 


83.0 


65.1 


87.6 


75.8 


years’’ 

Enrolled in at least one semester in each of the first three years or completed 


66.1 


46.4 


52.1 


44.8 


within 3 years’ 


36.5 


21.1 


24.5 


14.9 


Comnletions 

Completed within 3 years 


12.1 


9.7 


28.8 


26.6 


Obtained an associate degree within 3 years 


10.9 


8.4 


16.4 


15.6 


Obtained a certificate or diploma within 3 years 


1.4 


1.5 


11.6 


10.8 


Enrolled in at least one semester in the third year 


43.2 


26.9 


27.9 


17.7 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 25,587 students at 13 community colleges. Eigures represent average institutional rates. 
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Appendix Table B.7 (continued) 



“Pell Grant status was determined by receipt in any term in the three years. 

’’Grades of C or better must be earned to have completed a course successfully. 

“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

“'“Percent referred” is the percentage of students who were referred to developmental instruction in math. 

“The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

^“Percent referred” is the percentage of students who were referred to developmental instruction in English. 

®The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

’’Eor the fall 2004 cohort, the second year is academic year 2005-2006. 

'Eor the fall 2004 cohort, the third year is academic year 2006-2007. 
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Achieving the Dream: Community Colleges Count 



Appendix Table B,8 

Average Institutional Rates on Achieving the Dream Performance Indicators at Pennsylvania and Washington Colleges (Round 3), by 

Developmental Instruction Referral Status, Fall 2004 Cohort, Three- Year Outcomes 





PA Referred 


PA Not Referred 


WA Referred 


WANot 

Referred 


Gatekeeper courses 

Successful completion of gatekeeper math course^ 


27.7 


30.6 


24.5 


39.2 


Percent referred who enroll in gatekeeper math'’ 


27.4 


n/a 


26.5 


n/a 


Percent referred who complete gatekeeper math*’ 


21.7 


n/a 


23.8 


n/a 


Successful completion of gatekeeper English course'’ 


49.4 


44.7 


53.1 


46.0 


Percent referred who enroll in gatekeeper English'* 


45.1 


n/a 


51.7 


n/a 


Percent referred who complete gatekeeper English'* 


41.1 


n/a 


46.1 


n/a 


Course comuletiou 

Ratio of completed credits to attempted credits 


73.0 


82.2 


76.0 


81.8 
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Appendix Table B.8 (continued) 










WANot 




PA Referred 


PA Not Referred 


WA Referred 


Referred 


Persistence 










Enrolled in the first semester after the initial term of enrollment or completed 
within first year‘s 


72.1 


67.1 


80.6 


82.4 


Enrolled in at least one semester in the second year or completed within two years^ 


53.4 


49.7 


54.0 


49.2 


Enrolled in at least one semester in each of the first three years or completed within 
3 years® 


27.8 


22.2 


25.4 


15.3 


Comnletions 

Completed within 3 years 


8.7 


13.4 


18.4 


32.2 


Obtained an associate degree within 3 years 


7.4 


12.0 


14.7 


20.1 


Obtained a certificate or diploma within 3 years 


1.4 


1.6 


2.3 


12.2 


Enrolled in at least one semester in the third year 


33.2 


29.1 


29.2 


18.7 



SOURCE: CCRC calculations using the Achieving the Dream database. 

NOTES: Calculations for this table use all available data for sample members in the fall 2004 cohort at Pennsylvania and Washington Achieving the 
Dream colleges, which includes 25,587 students at 13 community colleges. Eigures represent average institutional rates. 
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Appendix Table B.8 (continued) 



“The gatekeeper math course is the first college-level math course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

’’“Percent referred” is the percentage of students who were referred to developmental instruction in math. The columns in the table for “Pennsylvania 
Not Referred” and “Washington Not Referred” are not applicable to the “Percent referred” measure. 

“The gatekeeper English course is the first college-level English course at the college. Grades of C or better must be earned to have completed a course 
successfully. 

“"“Percent referred” is the percentage of students who were referred to developmental instruction in English. The columns in the table for 
“Pennsylvania Not Referred” and “Washington Not Referred” are not applicable to the “Percent referred” measure. 

“The initial term of enrollment is fall 2004. The first term after the initial term is spring 2005. 

^Eor the fall 2004 cohort, the second year is academic year 2005-2006. 

®For the fall 2004 cohort, the third year is academic year 2006-2007. 
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Appendix C 

Achieving the Dream Colleges in 
Pennsylvania and Washington: 



Progress Toward Implementing Achieving the Dream 
Institutional Effectiveness Principles 




Achieving the Dream: Community Colleges Count 



Table C.l 

Achieving the Dream Colleges in Pennsylvania: 

Progress Toward Implementing Achieving the Dream Institutional Effectiveness Principles 



Rating Scale 

Extent to which a college implemented or developed practices that reflect the given principle: 



Little or None 




Developing 




A Lot 


1 


2 


3 


4 


5 



Principle 


PA-7 


PA -6 


PA-5 


PA-4 


PA-3 


PA-2 


PA-1 


1. Committed 
Leadership 
















Vision and values: 
College leaders actively 
support focus on student 
outcomes, not just 
enrollments 


4 


4 


5 


5 


5 


5 


5 
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Principle 


PA-7 


PA -6 


PA-5 


PA-4 


PA-3 


PA-2 


PA-1 


Equity: Leaders 
committed to achieving 
equity in outcomes 
across race/income 
groups 


3 


3 


2 


2 


3 


3 


3 


Commitment: 
Leadership willing to 
change policy and 
procedures, make 
investments to improve 
student success 


2 


2 


4 


4 


5 


5 


5 


2. Use of Evidence to 
Improve Policies, 
Programs and 
Services 
















IT capacity: IT capacity 
adequate to meet demand 
for data and IR 


5 


4 


2 


3 


4 


4 


4 


IR capacity: IR staff 
capacity adequate to 
meet demand 


5 


4 


2 


3 


4 


5 


5 


Process for identifying 
achievement gaps: 
College has an 
established process 


3 


4 


4 


4 


4 


4 


5 
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Principle 


PA-7 


PA -6 


PA-5 


PA-4 


PA-3 


PA-2 


PA-1 


Process for diagnosing 
gaps and formulating 
solutions: College has an 
established process 


2 


3 


4 


4 


4 


4 


5 


Process for evaluating 
solutions: College has an 
established process 

3. Broad Engagement 


2 


3 


3 


2 


3 


4 


5 


Faculty: Faculty actively 
involved in developing 
and assessing efforts to 
improve student success 


2 


2 


2 


3 


4 


4 


5 


Student services staff: 
Faculty actively involved 
in developing and 
assessing efforts to 
improve student success 


2 


2 


2 


3 


3 


4 


5 


Collaboration: Faculty 
and student services staff 
work together to improve 
student success 


2 


2 


2 


3 


3 


4 


4 


Students: Students 
actively participate in 
efforts to improve 
student success 


2 


3 


2 


3 


2 


3 


3 
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Principle 


PA-7 


PA -6 


PA-5 


PA-4 


PA-3 


PA-2 


PA-1 


External stakeholders: 
Colleges secure input 
from external 
stakeholders on efforts to 
improve student success 

4. Systemic 

Institutional 

Improvement 


3 


1 


3 


2 


3 


2 


2 


Institutional 
management: Program 
review, planning, and 
budgeting decisions 
driven by evidence on 
what works to improve 
student success 


2 


2 


3 


3 


4 


5 


5 


Organization: College 
has committee or body 
responsible for 
overseeing student 
success efforts 


2 


2 


4 


4 


3 


5 


5 


Professional 
development: 
Professional 
development for faculty 
and staff reinforce 
student success efforts 


3 


3 


4 


3 


3 


3 


4 
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Achieving the Dream: Community Colleges Count 

Table C.2 

Achieving the Dream Colleges in Washington State: 

Progress toward Implementing Achieving the Dream Institutional Effectiveness Principles 

Rating Scale 

Extent to which a college implemented or developed practices that reflect the given principle; 

Little or None Developing A Lot 

1 2 3 4 5 



Principle WA-6 WA-5 WA-4 WA-3 WA-2 WA-1 

1. Committed Leadership 

Vision and values: College leaders 3 3 4 4 5 5 

actively support focus on student 
outcomes, not just enrollments 

Equity: Leaders committed to 3 2 3 3 3 5 

achieving equity in outcomes across 
race/income groups 

Commitment: Leadership willing to 2 3 4 4 4 5 

change policy and procedures, make 

investments to improve student success 
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Principle 


WA-6 


WA-5 


WA-4 


WA-3 


WA-2 


WA-1 


2. Use of Evidence to Improve 
Policies, Programs and Services 














IT capacity: IT capacity adequate to 
meet demand for data and IR 


2 


2 


2 


2 


2 


2 


IR capacity: IR staff capacity adequate 
to meet demand 


3 


3 


3 


3 


4 


4 


Process for identifying achievement 
gaps: College has an established 
process 


2 


3 


4 


4 


5 


4 


Process for diagnosing gaps and 
formulating solutions: College has an 
established process 


2 


3 


3 


3 


5 


4 


Process for evaluating solutions: 
College has an established process 

3. Broad Engagement 


1 


3 


2 


3 


5 


5 


Faculty: Faculty actively involved in 
developing and assessing efforts to 
improve student success 


2 


3 


2 


3 


4 


5 


Student services staff: Faculty actively 
involved in developing and assessing 
efforts to improve student success 


2 


3 


2 


3 


4 


5 
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Principle 


WA-6 


WA-5 


WA-4 


WA-3 


WA-2 


WA-1 


Collaboration: Faculty and student 
services staff work together to improve 
student success 


2 


2 


2 


3 


4 


5 


Students: Students actively participate 
in efforts to improve student success 


3 


2 


2 


2 


4 


4 


External stakeholders: Colleges 
secure input from external stakeholders 
on efforts to improve student success 


2 


2 


2 


2 


2 


4 


4. Systemic Institutional 
Improvement 














Institutional management: Program 
review, planning, and budgeting 
decisions driven by evidence on what 
works to improve student success 


2 


2 


2 


2 


3 


4 


Organization: College has committee 
or body responsible for overseeing 
student success efforts 


3 


3 


2 


4 


4 


5 


Professional development: 
Professional development for faculty 
and staff reinforce student success 
efforts 


3 


3 


3 


3 


3 


3 
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